Discussion:
Unable to start DomR CS 4.1 VMware esxi 5.1
(too old to reply)
John Skinner
2013-09-16 20:56:10 UTC
Permalink
Having an issue getting domain routers to start in a VMware 5.1 environment with CS 4.1. The fun thing is, this was all working fine last week, and of course, no one changed anything . Has anyone seen anything like this:



2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for VM properties. target MOR: domain-c3887, properties: ["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) Configure VNC port for VM r-126-VM, port: 5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM

java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2315)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:419)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource] (DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the answers is false and it is stop on error.




Thats just the tail end of the thread for the start of the domr. Let me know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-16 23:08:09 UTC
Permalink
Found this in the esxi logs, not sure what it means. I am still not sure which "parameter" was not correct. Not sure if this is of any help or not.



2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788] [VpxLRO] -- FINISH task-46421 -- vm-4730 -- vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788] [VpxLRO] -- ERROR task-46421 -- vm-4730 -- vim.VirtualMachine.reconfigure: vmodl.fault.InvalidArgument:
--> Result:
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }

----- Original Message -----

From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1

Having an issue getting domain routers to start in a VMware 5.1 environment with CS 4.1. The fun thing is, this was all working fine last week, and of course, no one changed anything . Has anyone seen anything like this:



2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for VM properties. target MOR: domain-c3887, properties: ["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) Configure VNC port for VM r-126-VM, port: 5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM

java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2315)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:419)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource] (DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the answers is false and it is stop on error.




Thats just the tail end of the thread for the start of the domr. Let me know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-17 16:12:37 UTC
Permalink
Still working on this. Here is a full trace from vCenter logs. Still unsure what is going on.

http://fpaste.org/40151/

----- Original Message -----

From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Found this in the esxi logs, not sure what it means. I am still not sure which "parameter" was not correct. Not sure if this is of any help or not.



2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788] [VpxLRO] -- FINISH task-46421 -- vm-4730 -- vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788] [VpxLRO] -- ERROR task-46421 -- vm-4730 -- vim.VirtualMachine.reconfigure: vmodl.fault.InvalidArgument:
--> Result:
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }

----- Original Message -----


From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1

Having an issue getting domain routers to start in a VMware 5.1 environment with CS 4.1. The fun thing is, this was all working fine last week, and of course, no one changed anything . Has anyone seen anything like this:



2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for VM properties. target MOR: domain-c3887, properties: ["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) Configure VNC port for VM r-126-VM, port: 5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM

java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2315)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:419)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource] (DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the answers is false and it is stop on error.




Thats just the tail end of the thread for the start of the domr. Let me know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-17 20:50:07 UTC
Permalink
Looking at the catalina.out log from the management server I see this right before the error:

INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107) Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107) Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.

ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107) StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM

java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2315)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:419)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662) Unable to start VM on Host[-21-Routing] due to StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM

It looks like the last thing it was trying to do is configure the VNC port for the VM?

----- Original Message -----
From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Still working on this. Here is a full trace from vCenter logs. Still unsure what is going on.

http://fpaste.org/40151/

----- Original Message -----

From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Found this in the esxi logs, not sure what it means. I am still not sure which "parameter" was not correct. Not sure if this is of any help or not.



2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788] [VpxLRO] -- FINISH task-46421 -- vm-4730 -- vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788] [VpxLRO] -- ERROR task-46421 -- vm-4730 -- vim.VirtualMachine.reconfigure: vmodl.fault.InvalidArgument:
--> Result:
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }

----- Original Message -----


From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1

Having an issue getting domain routers to start in a VMware 5.1 environment with CS 4.1. The fun thing is, this was all working fine last week, and of course, no one changed anything . Has anyone seen anything like this:



2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for VM properties. target MOR: domain-c3887, properties: ["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO] (DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) Configure VNC port for VM r-126-VM, port: 5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource] (DirectAgent-69:10.0.97.107) StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM

java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2315)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:419)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource] (DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the answers is false and it is stop on error.




Thats just the tail end of the thread for the start of the domr. Let me know if you need more details. Any help would be appreciated. Thanks!
Kelven Yang
2013-09-17 21:16:42 UTC
Permalink
Yes, the last step is to configure VNC for the VM.

Are you able to start the VM in vCenter?

Kelven
Post by John Skinner
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareR
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.jav
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acces
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(S
cheduledThreadPoolExecutor.java:266)
at
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareR
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.jav
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acces
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(S
cheduledThreadPoolExecutor.java:266)
at
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-17 21:31:54 UTC
Permalink
Yes, however on the console it just comes up with "Operating System Not Found" error. This is really strange since this was all working fine last week.

John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing®

Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
john.skinner-sr0yiF+hT/***@public.gmane.org | www.appcore.com
----------------------------------------------------------------------
The information in this message is intended for the named recipients only. It may contain information that is privileged, confidential or otherwise protected from disclosure. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or the taking of any action in reliance on the contents of this message is strictly prohibited. If you have received this e-mail in error, do not print it or disseminate it or its contents. In such event, please notify the sender by return e-mail and delete the e-mail file immediately thereafter. Thank you.


----- Original Message -----
From: "Kelven Yang" <kelven.yang-Sxgqhf6Nn4DQT0dZR+***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Yes, the last step is to configure VNC for the VM.

Are you able to start the VM in vCenter?

Kelven
Post by John Skinner
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareR
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.jav
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acces
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(S
cheduledThreadPoolExecutor.java:266)
at
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareR
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.jav
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acces
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(S
cheduledThreadPoolExecutor.java:266)
at
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
Kelven Yang
2013-09-17 22:13:54 UTC
Permalink
It looks like that the disk image of the VM is corrupted. Do following,

1) Locate the datastore in vCenter on which the system VM is running
2) Open datastore-browse window
3) In the meantime, right click on the problem VM in vCenter (int host
view), chose "Edit SettingsŠ" to bring up VM configuration dialog
4) Find the VMDK location from the dialog in step 3
5) cross-check your findings about the VMDK in datastore, first check if
the file does exist on datastore

If it is a problem caused by disk corruption, you will need to destroy the
system and let CloudStack recreate it

Kelven
Post by John Skinner
Yes, however on the console it just comes up with "Operating System Not
Found" error. This is really strange since this was all working fine last
week.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing®
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Yes, the last step is to configure VNC for the VM.
Are you able to start the VM in vCenter?
Kelven
Post by John Skinner
Looking at the catalina.out log from the management server I see this
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResourc
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmware
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.ja
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
a
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still
unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResourc
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmware
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.ja
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
a
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-18 16:19:00 UTC
Permalink
I'll take a look at that. I am wondering if the problem is it is unable to pull the systemvm template. I noticed that the permissions for the systemvm .ova file on secondary storage have a different ownership then the rest of the files on secondary storage. Second thing I noticed, the template on the vCenter side has a network associated to it that is not be used by CloudStack and isn't routable in the network at all... and I am not sure how that happened.

John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing®

Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
john.skinner-sr0yiF+hT/***@public.gmane.org | www.appcore.com
----------------------------------------------------------------------
The information in this message is intended for the named recipients only. It may contain information that is privileged, confidential or otherwise protected from disclosure. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or the taking of any action in reliance on the contents of this message is strictly prohibited. If you have received this e-mail in error, do not print it or disseminate it or its contents. In such event, please notify the sender by return e-mail and delete the e-mail file immediately thereafter. Thank you.


----- Original Message -----
From: "Kelven Yang" <kelven.yang-Sxgqhf6Nn4DQT0dZR+***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Tuesday, September 17, 2013 5:13:54 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

It looks like that the disk image of the VM is corrupted. Do following,

1) Locate the datastore in vCenter on which the system VM is running
2) Open datastore-browse window
3) In the meantime, right click on the problem VM in vCenter (int host
view), chose "Edit SettingsŠ" to bring up VM configuration dialog
4) Find the VMDK location from the dialog in step 3
5) cross-check your findings about the VMDK in datastore, first check if
the file does exist on datastore

If it is a problem caused by disk corruption, you will need to destroy the
system and let CloudStack recreate it

Kelven
Post by John Skinner
Yes, however on the console it just comes up with "Operating System Not
Found" error. This is really strange since this was all working fine last
week.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing®
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Yes, the last step is to configure VNC for the VM.
Are you able to start the VM in vCenter?
Kelven
Post by John Skinner
Looking at the catalina.out log from the management server I see this
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResourc
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmware
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.ja
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
a
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still
unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResourc
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmware
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.ja
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
a
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
Kelven Yang
2013-09-18 17:05:49 UTC
Permalink
John,

Yes, the bottom of the disk chain for system VM lands to the template, if
for some reason it is not accessible, it will lead to corrupted disk
content and fail the VM bootstrap process

Kelven
Post by John Skinner
I'll take a look at that. I am wondering if the problem is it is unable
to pull the systemvm template. I noticed that the permissions for the
systemvm .ova file on secondary storage have a different ownership then
the rest of the files on secondary storage. Second thing I noticed, the
template on the vCenter side has a network associated to it that is not
be used by CloudStack and isn't routable in the network at all... and I
am not sure how that happened.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 5:13:54 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
It looks like that the disk image of the VM is corrupted. Do following,
1) Locate the datastore in vCenter on which the system VM is running
2) Open datastore-browse window
3) In the meantime, right click on the problem VM in vCenter (int host
view), chose "Edit SettingsŠ" to bring up VM configuration dialog
4) Find the VMDK location from the dialog in step 3
5) cross-check your findings about the VMDK in datastore, first check if
the file does exist on datastore
If it is a problem caused by disk corruption, you will need to destroy the
system and let CloudStack recreate it
Kelven
Post by John Skinner
Yes, however on the console it just comes up with "Operating System Not
Found" error. This is really strange since this was all working fine last
week.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Yes, the last step is to configure VNC for the VM.
Are you able to start the VM in vCenter?
Kelven
Post by John Skinner
Looking at the catalina.out log from the management server I see this
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still
unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-18 19:08:41 UTC
Permalink
Ok. So we stopped CloudStack management, changed the permissions on the .ova for the systemvm template, and removed the references from template_spool_ref to force CloudStack to send the template to vCenter again. We are still seeing this error when trying to start the Domain Router:


2013-09-18 13:59:29,732 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-42:10.0.97.105) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument


----- Original Message -----

From: "Kelven Yang" <kelven.yang-Sxgqhf6Nn4DQT0dZR+***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Wednesday, September 18, 2013 12:05:49 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

John,

Yes, the bottom of the disk chain for system VM lands to the template, if
for some reason it is not accessible, it will lead to corrupted disk
content and fail the VM bootstrap process

Kelven
Post by John Skinner
I'll take a look at that. I am wondering if the problem is it is unable
to pull the systemvm template. I noticed that the permissions for the
systemvm .ova file on secondary storage have a different ownership then
the rest of the files on secondary storage. Second thing I noticed, the
template on the vCenter side has a network associated to it that is not
be used by CloudStack and isn't routable in the network at all... and I
am not sure how that happened.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 5:13:54 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
It looks like that the disk image of the VM is corrupted. Do following,
1) Locate the datastore in vCenter on which the system VM is running
2) Open datastore-browse window
3) In the meantime, right click on the problem VM in vCenter (int host
view), chose "Edit SettingsÅ " to bring up VM configuration dialog
4) Find the VMDK location from the dialog in step 3
5) cross-check your findings about the VMDK in datastore, first check if
the file does exist on datastore
If it is a problem caused by disk corruption, you will need to destroy the
system and let CloudStack recreate it
Kelven
Post by John Skinner
Yes, however on the console it just comes up with "Operating System Not
Found" error. This is really strange since this was all working fine last
week.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Yes, the last step is to configure VNC for the VM.
Are you able to start the VM in vCenter?
Kelven
Post by John Skinner
Looking at the catalina.out log from the management server I see this
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still
unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-18 20:22:16 UTC
Permalink
Ok so here is something interesting. I just destroyed the console proxy system VM, and it recreated successfully. However, Domain routers will not create.


----- Original Message -----
From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Wednesday, September 18, 2013 2:08:41 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Ok. So we stopped CloudStack management, changed the permissions on the .ova for the systemvm template, and removed the references from template_spool_ref to force CloudStack to send the template to vCenter again. We are still seeing this error when trying to start the Domain Router:


2013-09-18 13:59:29,732 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-42:10.0.97.105) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument


----- Original Message -----

From: "Kelven Yang" <kelven.yang-Sxgqhf6Nn4DQT0dZR+***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Wednesday, September 18, 2013 12:05:49 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

John,

Yes, the bottom of the disk chain for system VM lands to the template, if
for some reason it is not accessible, it will lead to corrupted disk
content and fail the VM bootstrap process

Kelven
Post by John Skinner
I'll take a look at that. I am wondering if the problem is it is unable
to pull the systemvm template. I noticed that the permissions for the
systemvm .ova file on secondary storage have a different ownership then
the rest of the files on secondary storage. Second thing I noticed, the
template on the vCenter side has a network associated to it that is not
be used by CloudStack and isn't routable in the network at all... and I
am not sure how that happened.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 5:13:54 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
It looks like that the disk image of the VM is corrupted. Do following,
1) Locate the datastore in vCenter on which the system VM is running
2) Open datastore-browse window
3) In the meantime, right click on the problem VM in vCenter (int host
view), chose "Edit SettingsŠ" to bring up VM configuration dialog
4) Find the VMDK location from the dialog in step 3
5) cross-check your findings about the VMDK in datastore, first check if
the file does exist on datastore
If it is a problem caused by disk corruption, you will need to destroy the
system and let CloudStack recreate it
Kelven
Post by John Skinner
Yes, however on the console it just comes up with "Operating System Not
Found" error. This is really strange since this was all working fine last
week.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Yes, the last step is to configure VNC for the VM.
Are you able to start the VM in vCenter?
Kelven
Post by John Skinner
Looking at the catalina.out log from the management server I see this
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still
unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
John Skinner
2013-09-19 20:09:43 UTC
Permalink
My problem was some one has changed router.extra.public.nics in global config from 2 to 5. Once changed back to 2, routers launch just fine.


----- Original Message -----
From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Wednesday, September 18, 2013 3:22:16 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Ok so here is something interesting. I just destroyed the console proxy system VM, and it recreated successfully. However, Domain routers will not create.


----- Original Message -----
From: "John Skinner" <john.skinner-sr0yiF+hT/***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Wednesday, September 18, 2013 2:08:41 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

Ok. So we stopped CloudStack management, changed the permissions on the .ova for the systemvm template, and removed the references from template_spool_ref to force CloudStack to send the template to vCenter again. We are still seeing this error when trying to start the Domain Router:


2013-09-18 13:59:29,732 ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-42:10.0.97.105) VMware reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument


----- Original Message -----

From: "Kelven Yang" <kelven.yang-Sxgqhf6Nn4DQT0dZR+***@public.gmane.org>
To: users-***@public.gmane.org
Sent: Wednesday, September 18, 2013 12:05:49 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1

John,

Yes, the bottom of the disk chain for system VM lands to the template, if
for some reason it is not accessible, it will lead to corrupted disk
content and fail the VM bootstrap process

Kelven
Post by John Skinner
I'll take a look at that. I am wondering if the problem is it is unable
to pull the systemvm template. I noticed that the permissions for the
systemvm .ova file on secondary storage have a different ownership then
the rest of the files on secondary storage. Second thing I noticed, the
template on the vCenter side has a network associated to it that is not
be used by CloudStack and isn't routable in the network at all... and I
am not sure how that happened.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 5:13:54 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
It looks like that the disk image of the VM is corrupted. Do following,
1) Locate the datastore in vCenter on which the system VM is running
2) Open datastore-browse window
3) In the meantime, right click on the problem VM in vCenter (int host
view), chose "Edit SettingsŠ" to bring up VM configuration dialog
4) Find the VMDK location from the dialog in step 3
5) cross-check your findings about the VMDK in datastore, first check if
the file does exist on datastore
If it is a problem caused by disk corruption, you will need to destroy the
system and let CloudStack recreate it
Kelven
Post by John Skinner
Yes, however on the console it just comes up with "Operating System Not
Found" error. This is really strange since this was all working fine last
week.
John Skinner
Senior Systems Administrator | Appcore - the business of cloud computing(R)
Office +1.800.735.7104
Direct +1.515.612.7783 | Mobile +1.515.745.0248
----------------------------------------------------------------------
The information in this message is intended for the named recipients
only. It may contain information that is privileged, confidential or
otherwise protected from disclosure. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or the taking of any action in reliance on the contents of
this message is strictly prohibited. If you have received this e-mail in
error, do not print it or disseminate it or its contents. In such event,
please notify the sender by return e-mail and delete the e-mail file
immediately thereafter. Thank you.
----- Original Message -----
Sent: Tuesday, September 17, 2013 4:16:42 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Yes, the last step is to configure VNC for the VM.
Are you able to start the VM in vCenter?
Kelven
Post by John Skinner
Looking at the catalina.out log from the management server I see this
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Preparing NIC device on network cloud.public.2502.0.1-vSwitch2
INFO [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
Configure VNC port for VM r-134-VM, port: 5927, host: 10.0.97.107
Fault {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
Message A specified parameter was not correct.
ERROR [vmware.mo.VirtualMachineMO] (DirectAgent-317:10.0.97.107) VMware
reconfigVM_Task failed due to A specified parameter was not correct.
com.vmware.vim25.InvalidArgument
WARN [vmware.resource.VmwareResource] (DirectAgent-317:10.0.97.107)
StartCommand failed due to Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-134-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-21:job-662)
Unable to start VM on Host[-21-Routing] due to StartCommand failed due to
Exception: java.lang.Exception
Message: Failed to configure VM before start. vmName: r-134-VM
It looks like the last thing it was trying to do is configure the VNC port for the VM?
----- Original Message -----
Sent: Tuesday, September 17, 2013 11:12:37 AM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Still working on this. Here is a full trace from vCenter logs. Still
unsure what is going on.
http://fpaste.org/40151/
----- Original Message -----
Sent: Monday, September 16, 2013 6:08:09 PM
Subject: Re: Unable to start DomR CS 4.1 VMware esxi 5.1
Found this in the esxi logs, not sure what it means. I am still not sure
which "parameter" was not correct. Not sure if this is of any help or
not.
2013-09-16T10:22:29.858-05:00 [02968 info 'commonvpxLro' opID=8a6a6788]
[VpxLRO] -- FINISH task-46421 -- vm-4730 --
vim.VirtualMachine.reconfigure --
2013-09-16T10:22:29.859-05:00 [02968 info 'Default' opID=8a6a6788]
[VpxLRO] -- ERROR task-46421 -- vm-4730 --
--> (vmodl.fault.InvalidArgument) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> invalidProperty = <unset>,
--> msg = "A specified parameter was not correct.
--> ",
--> }
----- Original Message -----
Sent: Monday, September 16, 2013 3:56:10 PM
Subject: Unable to start DomR CS 4.1 VMware esxi 5.1
Having an issue getting domain routers to start in a VMware 5.1
environment with CS 4.1. The fun thing is, this was all working fine last
week, and of course, no one changed anything . Has anyone seen anything
2013-09-16 15:46:16,927 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties() for
["name","config.extraConfig[\"RemoteDisplay.vnc.port\"]"]
2013-09-16 15:46:17,222 TRACE [vmware.mo.ClusterMO]
(DirectAgent-69:10.0.97.107) vCenter API trace - retrieveProperties()
done
2013-09-16 15:46:17,451 INFO [vmware.resource.VmwareResource]
5905, host: 10.0.97.107
2013-09-16 15:46:18,931 ERROR [vmware.mo.VirtualMachineMO]
(DirectAgent-69:10.0.97.107) VMware reconfigVM_Task failed due to A
specified parameter was not correct.
com.vmware.vim25.InvalidArgument
2013-09-16 15:46:18,933 WARN [vmware.resource.VmwareResource]
java.lang.Exception
Message: Failed to configure VM before start. vmName: r-126-VM
java.lang.Exception: Failed to configure VM before start. vmName: r-126-VM
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResour
c
e
.java:2315)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(Vmwar
e
R
esource.java:419)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.j
a
v
a:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acc
e
s
s$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run
(
S
cheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
a
1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
v
a
:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-16 15:46:18,938 TRACE [vmware.resource.VmwareResource]
(DirectAgent-69:null) End executeRequest(), cmd: StartCommand
2013-09-16 15:46:18,938 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-69:null) Seq 21-2006712339: Cancelling because one of the
answers is false and it is stop on error.
Thats just the tail end of the thread for the start of the domr. Let me
know if you need more details. Any help would be appreciated. Thanks!
Loading...