Hi,
I've been running ArcGIS Server 10.1 with SP1 on a Windows Server 2012 64-bit machine for months without any problems in publishing map services and geoprocessing services. I have an ArcMap 32-bit installation on the same machine of the server, which I use to view maps and publish services on the server.
Yesterday, this crazy situation happened to me: I tried to publish a new map service through ArcMap, but it was stuck and said "Not responding" for minutes, so I killed it. Tried again and again: no fortune. Then I examined the server log file, which in my machine is at the path "C:\arcgisserver\logs\<machine_name>\server" and I found lines like the following ones:
Code:
< ... type="SEVERE" code="8273" source="Server" process="3192" thread="1" methodName="" ... user="" elapsed="">Services containing process crashed. Utilities/PrintingTools.GPServer instance has crashed.</Msg>
< ... type="SEVERE" code="8273" source="Server" process="3192" thread="1" methodName="" ... user="" elapsed="">Services containing process crashed. System/PublishingTools.GPServer instance has crashed.</Msg>
The log file contained one crash-report line for each of the geoprocessing services running on the server.
I therefore tried to investigate: I opened the ArcGIS Manager web application and tried to restart the PublishingTools service (which comes bundled with the server): it took a lot of time, then the service correctly stopped but in the end this message popped up:
Code:
System.PublishingTools.GPServer: java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is: java.net.SocketException: Connection reset
I tried to restart all of the other geoprocessing (either System's and my custom ones), and for each of them I got similar messages printed on screen.
I then tried to restart all of the other services (map services, geometry services) and everything went fine!
So it seems to be the just an error on the geoprocessing services.
I further investigated by setting the server log's verbosity to DEBUG (that is: print everything!) and this is the full stack-trace of the exception:
Code:
<Msg time="..." type="SEVERE" code="8273" source="Server" process="4664" thread="1" methodName="" machine="..." user="" elapsed="">Services containing process crashed. System/PublishingTools.GPServer instance has crashed.</Msg>
<Msg time="..." type="DEBUG" code="9999" source="Server" process="4664" thread="1" methodName="" machine="..." user="" elapsed="">java.lang.Exception: System/PublishingTools.GPServer instance has crashed.
at com.esri.arcgis.discovery.ejb.util.EJBBase.initService(EJBBase.java:363)
at com.esri.arcgis.discovery.ejb.util.EJBBase.init(EJBBase.java:218)
at com.esri.arcgis.discovery.ejb.impl.GPServerSyncBean.init(GPServerSyncBean.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:162)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext$LifecycleInvocation.invoke(ReflectionInvocationContext.java:194)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:144)
at org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:164)
at org.apache.openejb.monitoring.StatsInterceptor.PostConstruct(StatsInterceptor.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:162)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:144)
at org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:122)
at org.apache.openejb.core.stateless.StatelessInstanceManager.ceateInstance(StatelessInstanceManager.java:237)
at org.apache.openejb.core.stateless.StatelessInstanceManager.createInstance(StatelessInstanceManager.java:448)
at org.apache.openejb.core.stateless.StatelessInstanceManager.access$100(StatelessInstanceManager.java:72)
at org.apache.openejb.core.stateless.StatelessInstanceManager$BackgroundThread.run(StatelessInstanceManager.java:507)
Caused by: java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:
java.net.SocketException: Connection reset
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:209)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:142)
at sun.rmi.server.ActivatableRef.invoke(ActivatableRef.java:124)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:178)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:132)
at $Proxy43.createService(Unknown Source)
at com.esri.arcgis.discovery.ejb.util.EJBBase.initService(EJBBase.java:340)
... 22 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.DataInputStream.readByte(DataInputStream.java:248)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:195)
... 28 more
</Msg>
As additional information:
- All firewall systems on my machine are fully deactivated
- No recent modification has been made to the networking in my LAN
- I installed a few Windows updates last week
Can anyone help me in solving the issue?
Another question: could this problem be tied to a "dirty" Python installation? I'm not sure that GP services rely on Python (I'm getting a Java exception!)