Hi guys, I was hoping you might be able to help me troubleshoot this issue I'm having. We have 5 identical servers running our application. They are all configured identically. About a week ago, web1, our first node, started locking up a few times a day, with the coldfusion process jacked up at 100% cup usage. I have verified the settings are identical to teh other 4 boxes, and I have copied over a fresh copy of the codebase. I have also applied all updates on the misbehaving box, as well as updateing to CF8.01 hotfix 4, and JRE 1.6.0_23. I have cleared all cache, and I have rebooted the machine several times. The hangups are still happening, seemingly at random.
Here's the specs on the machines:
Dell 1950's, Two dual core 3.0 GHz xeon's, 8GB memory, two 147GB 15k scsi in raid 1.
Cent OS 5.5 64 Bit
Coldfusion 8.01 Standard (32 bit)
Apache 2.2.3 (32 bit)
Here are some log file snippets from when the machine is experiencing problems:
CF application log
"Error","jrpp-40","02/10/11","09:31:51","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
"Error","jrpp-35","02/10/11","09:31:51",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-31","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-30","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
"Error","jrpp-9","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-51","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
"Error","jrpp-49","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-11","02/10/11","09:35:21","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
"Error","jrpp-21","02/10/11","09:50:14","SO","The request has exceeded the allowable time limit Tag: CFQUERY The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 247 "
CF Exception log
"Error","jrpp-40","02/10/11","09:31:51","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45)
at java.lang.StringBuilder.<init>(StringBuilder.java:68)
at java.io.UnixFileSystem.resolve(UnixFileSystem.java:93)
at java.io.File.<init>(File.java:207)
at java.io.File.listFiles(File.java:1056)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:54)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:62)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:48)
at coldfusion.compiler.CustomTagTable.<init>(CustomTagTable.java:34)
at coldfusion.runtime.RuntimeServiceImpl.resolveTemplateName(RuntimeServ iceImpl.java:579)
at coldfusion.tagext.lang.ModuleTag.setName(ModuleTag.java:84)
at coldfusion.tagext.lang.CustomTag.setName(CustomTag.java:34)
at cfdsp_edit2ecfm45629791._factor5(/san/html/sourceone/admin/dsp_edit.c fm:39)
at cfdsp_edit2ecfm45629791._factor14(/san/html/sourceone/admin/dsp_edit. cfm:37)
at cfdsp_edit2ecfm45629791.runPage(/san/html/sourceone/admin/dsp_edit.cf m:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Switch2ecfm1533486892._factor41(/san/html/sourceone/admin/fbx_S witch.cfm:346)
at cffbx_Switch2ecfm1533486892._factor109(/san/html/sourceone/admin/fbx_ Switch.cfm:340)
at cffbx_Switch2ecfm1533486892.runPage(/san/html/sourceone/admin/fbx_Swi tch.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor4(/san/html/sourceone/ fbx_fusebox30_CF50_nix.cfm:242)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor5(/san/html/sourceone/ fbx_fusebox30_CF50_nix.cfm:1)
at cffbx_fusebox30_CF50_nix2ecfm1062751667.runPage(/san/html/sourceone/f bx_fusebox30_CF50_nix.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1792679210.runPage(/san/html/sourceone/index.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
"Error","jrpp-35","02/10/11","09:31:51",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_Glob alRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cf m:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings. cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:159)
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fus ebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-31","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_Glob alRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cf m:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings. cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:159)
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fus ebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter. java:126)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-30","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings. cfm:61)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:159)
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fus ebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-9","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_Glob alRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cf m:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings. cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:159)
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fus ebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-51","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings. cfm:61)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:159)
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fus ebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-49","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_Glob alRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cf m:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings. cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:159)
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fu sebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fus ebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-11","02/10/11","09:35:21","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.io.File.listFiles(File.java:1056)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:54)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:62)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:48)
at coldfusion.compiler.CustomTagTable.<init>(CustomTagTable.java:34)
at coldfusion.runtime.RuntimeServiceImpl.resolveTemplateName(RuntimeServ iceImpl.java:579)
at coldfusion.tagext.lang.ModuleTag.setName(ModuleTag.java:84)
at coldfusion.tagext.lang.CustomTag.setName(CustomTag.java:34)
at cfdsp_edit2ecfm45629791._factor5(/san/html/sourceone/admin/dsp_edit.c fm:39)
at cfdsp_edit2ecfm45629791._factor14(/san/html/sourceone/admin/dsp_edit. cfm:37)
at cfdsp_edit2ecfm45629791.runPage(/san/html/sourceone/admin/dsp_edit.cf m:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Switch2ecfm1533486892._factor41(/san/html/sourceone/admin/fbx_S witch.cfm:346)
at cffbx_Switch2ecfm1533486892._factor109(/san/html/sourceone/admin/fbx_ Switch.cfm:340)
at cffbx_Switch2ecfm1533486892.runPage(/san/html/sourceone/admin/fbx_Swi tch.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor4(/san/html/sourceone/ fbx_fusebox30_CF50_nix.cfm:242)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor5(/san/html/sourceone/ fbx_fusebox30_CF50_nix.cfm:1)
at cffbx_fusebox30_CF50_nix2ecfm1062751667.runPage(/san/html/sourceone/f bx_fusebox30_CF50_nix.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1792679210.runPage(/san/html/sourceone/index.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
"Error","jrpp-21","02/10/11","09:50:14","SO","The request has exceeded the allowable time limit Tag: CFQUERY The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 247 "
coldfusion.runtime.CustomException: The request has exceeded the allowable time limit Tag: CFQUERY
at coldfusion.tagext.lang.ThrowTag.doStartTag(ThrowTag.java:124)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor4(/san/html/sourceone/ fbx_fusebox30_CF50_nix.cfm:247)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor5(/san/html/sourceone/ fbx_fusebox30_CF50_nix.cfm:1)
at cffbx_fusebox30_CF50_nix2ecfm1062751667.runPage(/san/html/sourceone/f bx_fusebox30_CF50_nix.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1792679210.runPage(/san/html/sourceone/index.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279 )
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.ja va:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:7 4)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePers istenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:8 9)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringS ervletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46 )
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java: 286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java: 543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.ja va:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPoo l.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.j ava:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool. java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
Any suggestions on where to look for clues would be greatly appreciated.