Don't wait until you have a server crash. Be proactive and analyze your servers on a regular basis.
Step 1: Analyze Log Files - check the following log files for irregularities
If you have a server problem, it is beneficial to have a "snapshot" of all the log files so you can have an organized data source for locating where your problem is.
- (cf)\logs
- (cf)\runtime\logs(JRun Logs)
- web server logs
- database server logs
- windows event viewer
- windows performance monitor and logging options
- equivalents on other OS platforms
- monitor log files (ex. FusionReactor log files)
Step 2: Monitor your Server - keep requests and memory in control
Server monitors allow you to look inside your server to view memory, CPU, requests and more. The leading ColdFusion server monitor is FusionReactor. FusionReactor supports Adobe ColdFusion 6, 7 and 8, LiveCycle Data Services, J2EE Application servers, and 3rd party CFML engines such as Railo and BlueDragon.
Step 3: Analyze your Problem
- Visualize the problem
- Graph monitoring tool log-file data (ex: graphing FusionReactor data in excel)
- Connect the dots
- Spot patterns
- Test the theory
- Fix the problem
Common Server Problems that can be addressed with monitoring tools like FusionReactor
Problem: ColdFusion runs out of memory
Solution: Use monitoring tools like FusionReactor to view memory for the server
Why would I want to see a memory graph?
Because of its visual nature, memory graphs can very quickly give you a good idea of how your system is performing. Since a memory graph can cover larger periods of time, it can warn you of trends which could become a problem in the future. This kind of trend is much more difficult to spot if you are only checking the system periodically and are only looking at the health of the system right at a specific point in time.
Problem: ColdFusion stops responding
Solution: Use monitoring tools to see what requests are running
The FusionReactor stored requests hold much more information than you would normally expect to find in web logs and as such they have a much higher value for analyzing your application traffic.
The Longest Requests table has the following columns:
| Finished | The time at which the request was completed. |
| Ret.Code/IP | The final HTTP return code of the request and the IP address of the caller. If the request was killed or if it triggered Crash Protection then you will see a note here letting you know what happened. |
| ID | The request ID and the name of the thread which is servicing the request. |
| URL/Parameters | The URL path and parameters passed to the request. You can turn on or off the hostname in the URL by modifying the URL Format value on the Request Settings page. |
| Time (ms) | The number of milliseconds that the request took to complete. If available then you will also see the amount of actual CPU time that this request required. |
| Memory (KB) | A snapshot of the state of system memory at the time the request was started. The following values are shown: Used memory as a percentage of the total available to the JVM, the actual used memory value in KB, and the margin of free memory available before the JVM must extend its own heap (if it has not already reached its limit). |
Problem: Requests taking too long
Solution: Use monitoring tools to view details of request. In FusionReactor view the request stack trace.
Why would I want to see a stack trace?
Stack Trace gives you valuable information as to what is actually happening inside your server. You can isolate long running requests to find out exactly what kind of activity they are engaged in. File access? Database access? Or maybe just waiting for some other resource to become free. You can refresh a stack trace and find out if a request is being worked on or if it looks like it is just going to wait forever. For development it is useful to see exactly what a request is doing in order to be able to optimize that request. For support, stack trace is an invaluable way of determining if a request is really hung before making a decision to kill it.
Problem: (Native method) in your stack tracks / I/O socket activities
Ex: java.net.SocketOutputStream.socketWrite0(Native Method)
Solution: Prevent (native method) issues from happening
It's not possible to kill a thread in Java that is in a Native method, the only way to stop the thread is to restart the Java runtime (i.e. restart ColdFusion), unfortunately this is a limitation of Java itself.
Typically it appears from the rest of the stack trace that the page has finished processing and is returning the remaining content to the client. Via FusionReactor a defect in Internet Explorer has been found.
It is known bug that Internet Explorer sometimes keeps connections open but does not read the data from the connection, even if you close the window or surf away to another page. This leads to the problem that Java can end up with stuck threads that never finish and cannot be killed because they are in native code.
Related Tech-note:When I kill a process, it times out and remains in the list as "Killing". Shouldn't this kill it immediately?
Continuous Server Analysis - Avoid the problems - This REALLY works!
Daily: Monitor your server applications with monitoring tools like FusionReactor.
Weekly: Examine log files for unexpected errors and resolve all unexpected errors.
Monthly: Examine server performance metrics to identify trends (slowest pages overall, slowest queries) and improve the performance of the top 3 slowest pages and queries.
Quarterly: Track metrics for capacity planning, average CPI Usage, memory usage, number of pages run and peak load. Prepare for any infrastructure requirement changes.
Yearly: Review the direction and need for software solutions with Dev Team, Support Team, Managers, System Sponsors. Develop an action plan for the next year.
More Information
More Information: Quick Start to Trouble Shooting Server Problems(pdf)
Download Free Trial: FusionReactor, the leading CF Server Monitor
Contact us: for ColdFusion consulting or for additional information
