Hi guys, I was hoping you might be able to help me troubleshoot this issue I'm having. We have 5 identical servers running our application. They are all configured identically. About a week ago, web1, our first node, started locking up a few times a day, with the coldfusion process jacked up at 100% cup usage. I have verified the settings are identical to teh other 4 boxes, and I have copied over a fresh copy of the codebase. I have also applied all updates on the misbehaving box, as well as updateing to CF8.01 hotfix 4, and JRE 1.6.0_23. I have cleared all cache, and I have rebooted the machine several times. The hangups are still happening, seemingly at random.
Here's the specs on the machines:
Dell 1950's, Two dual core 3.0 GHz xeon's, 8GB memory, two 147GB 15k scsi in raid 1.
Cent OS 5.5 64 Bit
Coldfusion 8.01 Standard (32 bit)
Apache 2.2.3 (32 bit)
Here are some log file snippets from when the machine is experiencing problems:
CF application log
"Error","jrpp-40","02/10/11","09:31:51","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
"Error","jrpp-35","02/10/11","09:31:51",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-31","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-30","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
"Error","jrpp-9","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-51","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
"Error","jrpp-49","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
"Error","jrpp-11","02/10/11","09:35:21","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
"Error","jrpp-21","02/10/11","09:50:14","SO","The request has exceeded the allowable time limit Tag: CFQUERY The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 247 "
CF Exception log
"Error","jrpp-40","02/10/11","09:31:51","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45)
at java.lang.StringBuilder.<init>(StringBuilder.java:68)
at java.io.UnixFileSystem.resolve(UnixFileSystem.java:93)
at java.io.File.<init>(File.java:207)
at java.io.File.listFiles(File.java:1056)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:54)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:62)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:48)
at coldfusion.compiler.CustomTagTable.<init>(CustomTagTable.java:34)
at coldfusion.runtime.RuntimeServiceImpl.resolveTemplateName(RuntimeServiceImpl.java:579)
at coldfusion.tagext.lang.ModuleTag.setName(ModuleTag.java:84)
at coldfusion.tagext.lang.CustomTag.setName(CustomTag.java:34)
at cfdsp_edit2ecfm45629791._factor5(/san/html/sourceone/admin/dsp_edit.cfm:39)
at cfdsp_edit2ecfm45629791._factor14(/san/html/sourceone/admin/dsp_edit.cfm:37)
at cfdsp_edit2ecfm45629791.runPage(/san/html/sourceone/admin/dsp_edit.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Switch2ecfm1533486892._factor41(/san/html/sourceone/admin/fbx_Switch.cfm:346)
at cffbx_Switch2ecfm1533486892._factor109(/san/html/sourceone/admin/fbx_Switch.cfm:340)
at cffbx_Switch2ecfm1533486892.runPage(/san/html/sourceone/admin/fbx_Switch.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor4(/san/html/sourceone/fbx_fusebox30_CF50_n ix.cfm:242)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor5(/san/html/sourceone/fbx_fusebox30_CF50_n ix.cfm:1)
at cffbx_fusebox30_CF50_nix2ecfm1062751667.runPage(/san/html/sourceone/fbx_fusebox30_CF50_ni x.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1792679210.runPage(/san/html/sourceone/index.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
"Error","jrpp-35","02/10/11","09:31:51",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_GlobalRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings.cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fusebox30_CF50.cfm:159 )
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-31","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_GlobalRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings.cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fusebox30_CF50.cfm:159 )
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-30","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings.cfm:61)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fusebox30_CF50.cfm:159 )
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-9","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_GlobalRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings.cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fusebox30_CF50.cfm:159 )
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-51","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 61 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings.cfm:61)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fusebox30_CF50.cfm:159 )
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-49","02/10/11","09:31:52",,"'' The specific sequence of files included or processed is: /san/html/dynamic/index.cfm, line: 18 "
java.lang.NullPointerException
at coldfusion.util.Utils.getDirectoryFromPath(Utils.java:294)
at coldfusion.runtime.CFPage.GetDirectoryFromPath(CFPage.java:2909)
at cfapp_GlobalRequests2ecfm789074969.runPage(/san/html/dynamic/app_GlobalRequests.cfm:18)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfapp_Globals2ecfm2092923734.runPage(/san/html/dynamic/app_Globals.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Settings2ecfm2009881526.runPage(/san/html/dynamic/fbx_Settings.cfm:79)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF502ecfm1499241541._factor3(/san/html/dynamic/fbx_fusebox30_CF50.cfm:159 )
at cffbx_fusebox30_CF502ecfm1499241541._factor6(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at cffbx_fusebox30_CF502ecfm1499241541.runPage(/san/html/dynamic/fbx_fusebox30_CF50.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1943790857.runPage(/san/html/dynamic/index.cfm:20)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
"Error","jrpp-11","02/10/11","09:35:21","SO","GC overhead limit exceeded The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 39 "
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.io.File.listFiles(File.java:1056)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:54)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:62)
at coldfusion.compiler.CustomTagTable.scanTags(CustomTagTable.java:48)
at coldfusion.compiler.CustomTagTable.<init>(CustomTagTable.java:34)
at coldfusion.runtime.RuntimeServiceImpl.resolveTemplateName(RuntimeServiceImpl.java:579)
at coldfusion.tagext.lang.ModuleTag.setName(ModuleTag.java:84)
at coldfusion.tagext.lang.CustomTag.setName(CustomTag.java:34)
at cfdsp_edit2ecfm45629791._factor5(/san/html/sourceone/admin/dsp_edit.cfm:39)
at cfdsp_edit2ecfm45629791._factor14(/san/html/sourceone/admin/dsp_edit.cfm:37)
at cfdsp_edit2ecfm45629791.runPage(/san/html/sourceone/admin/dsp_edit.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_Switch2ecfm1533486892._factor41(/san/html/sourceone/admin/fbx_Switch.cfm:346)
at cffbx_Switch2ecfm1533486892._factor109(/san/html/sourceone/admin/fbx_Switch.cfm:340)
at cffbx_Switch2ecfm1533486892.runPage(/san/html/sourceone/admin/fbx_Switch.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor4(/san/html/sourceone/fbx_fusebox30_CF50_n ix.cfm:242)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor5(/san/html/sourceone/fbx_fusebox30_CF50_n ix.cfm:1)
at cffbx_fusebox30_CF50_nix2ecfm1062751667.runPage(/san/html/sourceone/fbx_fusebox30_CF50_ni x.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1792679210.runPage(/san/html/sourceone/index.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
"Error","jrpp-21","02/10/11","09:50:14","SO","The request has exceeded the allowable time limit Tag: CFQUERY The specific sequence of files included or processed is: /san/html/sourceone/index.cfm, line: 247 "
coldfusion.runtime.CustomException: The request has exceeded the allowable time limit Tag: CFQUERY
at coldfusion.tagext.lang.ThrowTag.doStartTag(ThrowTag.java:124)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor4(/san/html/sourceone/fbx_fusebox30_CF50_n ix.cfm:247)
at cffbx_fusebox30_CF50_nix2ecfm1062751667._factor5(/san/html/sourceone/fbx_fusebox30_CF50_n ix.cfm:1)
at cffbx_fusebox30_CF50_nix2ecfm1062751667.runPage(/san/html/sourceone/fbx_fusebox30_CF50_ni x.cfm:1)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2661)
at cfindex2ecfm1792679210.runPage(/san/html/sourceone/index.cfm:11)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:196)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:370)
at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:279)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:86)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:74)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:2 8)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.CfmServlet.service(CfmServlet.java:175)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42 )
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
Any suggestions on where to look for clues would be greatly appreciated.