oprofile
This is a sampling profiler, so it has minimal impact on Traffic Servers performance.
...
Running a basic report will tell you what functions are showing up in the sampled profile the most.
| Panel |
|---|
bcall@snowball trafficserver$ sudo opreport --symbols /usr/local/bin/traffic_server | head
CPU: Core 2, speed 2003 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 100000
samples % image name app name symbol name
5158 4.6037 traffic_server traffic_server ink_freelist_new
3223 2.8767 traffic_server traffic_server HttpSM::_instantiate_func(HttpSM*, HttpSM*)
3060 2.7312 traffic_server traffic_server mime_hdr_field_find(MIMEHdrImpl*, char const*, int)
3044 2.7169 traffic_server traffic_server ink_freelist_free
2586 2.3081 libc-2.11.1.so traffic_server memcpy
2487 2.2197 traffic_server traffic_server HttpCompat::parse_tok_list(StrList*, int, char const*, int, char)
2309 2.0609 libc-2.11.1.so traffic_server vfprintf |
Running a callgraph report.
| Panel |
|---|
bcall@snowball trafficserver$ sudo opreport -cg --symbols /usr/local/bin/traffic_server 2 | head -50
samples % linenr info image name app name symbol name
-------------------------------------------------------------------------------
18 26.4706 HttpSM.cc:2640 traffic_server traffic_server HttpSM::main_handler(int, void*)
20 29.4118 HttpAccept.cc:33 traffic_server traffic_server HttpAccept::mainEvent(int, void*)
30 44.1176 HttpClientSession.cc:428 traffic_server traffic_server HttpClientSession::state_api_callout(int, void*)
5158 4.6037 ink_queue.cc:169 traffic_server traffic_server ink_freelist_new
5158 100.000 ink_queue.cc:169 traffic_server traffic_server ink_freelist_new self
-------------------------------------------------------------------------------
26 100.000 HttpSM.cc:2640 traffic_server traffic_server HttpSM::main_handler(int, void*)
3223 2.8767 HttpSM.cc:154 traffic_server traffic_server HttpSM::_instantiate_func(HttpSM*, HttpSM*)
3223 100.000 HttpSM.cc:154 traffic_server traffic_server HttpSM::_instantiate_func(HttpSM*, HttpSM*) self
-------------------------------------------------------------------------------
3060 2.7312 MIME.cc:1380 traffic_server traffic_server mime_hdr_field_find(MIMEHdrImpl*, char const*, int)
3060 100.000 MIME.cc:1380 traffic_server traffic_server mime_hdr_field_find(MIMEHdrImpl*, char const*, int) self |
Running a callgraph report.
Google profiler
This too is a sampling profiler, so it has minimal impact on Traffic Server's performance. Support for the profiler needs to be compiled into Traffic Server for it to work. There is an option to add profiling support when running configure and is described in the steps below.
Install the Google's perftools development package, the command may vary on the OS, below is on Fedora 12:
| Panel |
|---|
bcall@snowball trafficserver$ opreport -c -g --symbols proxy/traffic_server
samples % linenr info image name app name symbol name
-------------------------------------------------------------------------------
10 1.3661 UnixConnection.cc:162 traffic_server traffic_server Connection::bind_connect(unsigned int, int, unsigned int, NetVCOptions*, int, bool, bool, bool, bool, bool)
68 9.2896 Update.cc:1138 traffic_server traffic_server UpdateScheduler::ScheduleEvent(int, void*)
71 9.6995 Hash_Table.cc:223 traffic_server traffic_server HashTable::remove(int, char*, _alt_hdr*)
223 30.4645 UnixNetAccept.cc:166 traffic_server traffic_server NetAccept::allocateThread(EThread*)
360 49.1803 HttpSM.cc:7791 traffic_server traffic_server HttpSM::redirect_request(char const*, int)
36603 8.1552 UnixEThread.cc:41 traffic_server traffic_server EThread::EThread()
36603 100.000 UnixEThread.cc:41 traffic_server traffic_server EThread::EThread() self
-------------------------------------------------------------------------------
13724 3.0577 LogBuffer.cc:356 traffic_server traffic_server LogBuffer::checkout_write(unsigned long*, unsigned long)
13724 100.000 LogBuffer.cc:356 traffic_server traffic_server LogBuffer::checkout_write(unsigned long*, unsigned long) self
-------------------------------------------------------------------------------
11137 2.4813 (no location information) libpthread-2.11.1.so traffic_server pthread_mutex_trylock
11137 100.000 (no location information) libpthread-2.11.1.so traffic_server pthread_mutex_trylock self |
Google profiler
...
sudo yum install google-perftools-devel
...
Installed:
google-perftools-devel.x86_64 0:1.4-1.fc12 Dependency Installed:
google-perftools.x86_64 0:1.4-1.fc12 Complete! |
Run configure with the option to add profiling support. It will check to see if the header and library is installed. You will have to recompile traffic server to add calling the profiler's start and stop functions.
| Panel |
|---|
bcall@snowball trafficserver$ ./configure --with-profiler
bcall@snowball trafficserver$ gmake -j >& /dev/null && echo good
good |
You can either install the package with "sudo gmake install" or run the program for the current directory if you already have an installation with all the logs and configuration directories setup.
| Panel |
|---|
bcall@snowball trafficserver$ sudo proxy/traffic_server
// or
bcall@snowball trafficserver$ sudo gmake install
bcall@snowball trafficserver$ sudo /usr/local/bin/trafficserver start
Starting Apache Traffic Server: [ OK ] |
Next you will need to run your benchmark. After the benchmark is complete, kill the traffic_server process or stop traffic server with the startup script. The traffic server process will need to be shutdown before a proper profile file is written.
| Panel |
|---|
bcall@snowball trafficserver$ sudo killall traffic_server
// or
bcall@snowball trafficserver$ sudo /usr/local/bin/trafficserver stop
Stopping traffic_cop: [ OK ]
Stopping traffic_manager: [ OK ]
Stopping traffic_server: [ OK ] |
Now we can look at the profiling information.
| Panel |
|---|
bcall@snowball trafficserver$ pprof --text proxy/traffic_server /tmp/ts.prof | head
Total: 14 samples
5 35.7% 35.7% 5 35.7% memset
3 21.4% 57.1% 3 21.4% __strncpy_ssse3
1 7.1% 64.3% 1 7.1% __pread_nocancel
1 7.1% 71.4% 1 7.1% build_part_hash_table
1 7.1% 78.6% 1 7.1% initialize_all_global_stats
1 7.1% 85.7% 1 7.1% ink_freelist_new
1 7.1% 92.9% 1 7.1% pthread_sigmask
1 7.1% 100.0% 1 7.1% time
0 0.0% 100.0% 2 14.3% AIOCallbackInternal::io_complet |
To make a callgraph of the program in PDF form.
| Panel |
|---|
bcall@snowball trafficserver$ pprof --pdf proxy/traffic_server /tmp/ts.prof > ts.pdf |
If you want to investigate a certain function you can the focus option.
| Panel |
|---|
bcall@snowball trafficserver$ pprof --pdf --focus=HttpSM::main_handler proxy/traffic_server /tmp/ts.prof > ts-main_handler.pdf |
Callgrind
Callgrind is will slow down the performance of traffic server significantly.
