Alois Kraus

blog

  Home  |   Contact  |   Syndication    |   Login
  133 Posts | 8 Stories | 368 Comments | 162 Trackbacks

News



Archives

Image Galleries

Programming

Finding handle leaks in all processes at once for all handle types without a debugger is no longer impossible. Since Windows 8.1 (0?) each handle creation and close call is instrumented with an ETW event. You only need to turn it on, execute your use case for some minutes or hours if you really need to and then stop the recording.

To start full handle tracing you need to install the Windows Performance Toolkit from the Windows 10 SDK or WDK. Then enter in an Administrator shell

  • wpr -start Handle
  • Execute your use case
  • wpr -stop c:\temp\Handle.etl

Then you can open the resulting .ETL file with WPA and add the graph Handles - Outstanding Count by Process to your analysis view.

image

Now you can filter for your process (e.g. in my case I did start Visual Studio). The original view gives me a system wide view of all processes which did allocate handles.

image

That is a nice view but if you are after a handle leak you need to Create Stack. No problem. Right click on the table header and add the Create Stack to the column list. Then you should load the symbol from MS and add your local symbol paths

image

With the call stacks you can drill into the allocation stack of any handle and search for your leak:

image

The graph nicely shows the not yet freed handles but the table shows all allocations which can be a bit confusing when you search for the not yet released handles. For big handle leaks the existing view is already enough but if you need in the table to drill down only into call stacks of not yet released handles you need to add a filter to exclude all lines in the table which have released a handle before the trace was stopped.

More Details

To add that filter click on the open the gear icon or press Ctrl+E:

image

Because we are doing advanced things we click on the Advanced icon

image

and there we can finally add the trace end time which is visible at the bottom of the WPA main window

image

Now the graph and the table is updated which now only shows the handles which have not been released since the start of Visual Studio in our example which should match the number of allocated handles shown by Task Manager.

image

You can also get more fancy. Normally I have some test which shows after some time a handle leak in a specific process. I start leak tracing and then the test and later I stop it. Since I do not want to treat first time initialization effects as leaks I can exclude the e.g. first 5 minutes of the test to get rid of first time init effects. I also want to make sure that I do not get handles as leaks which are allocated at the end because the test was still running at the end of the trace. To do that I need to look for recurring patterns in the trace and exclude all allocated handles which were created at some later time when the test run was just complete. The final result is a filter which hides all entries which match

[Close Time]:<"20,861s" OR [Create Time]:<"5s" OR [Create Time]:>"15s"

After all noise is removed any handle leak, even small ones are only a matter of drilling into the allocation call stacks and fixing the code. If you have a handle leak on a Windows 8.1 (0?) or later machine this approach is much easier and faster than to use Windbg and the !htrace command which is nicely explained at https://blogs.technet.microsoft.com/yongrhee/2011/12/19/how-to-troubleshoot-a-handle-leak/.

Why So Late?

I have no idea why this very useful capability of WPA was never documented anywhere. It showed up in the Windows 8 SDK years ago but Handle leak tracing did never work because I was at that time still with Windows 7.

Which Handle Type did I Leak?

The easiest way is to use another tool. Process Hacker is a Process Explorer clone which can show for any process a nice summary. Double click on a process and select the Statistics tab:

image

When you click on Details you can sort by Handle Count and you immediately know for which handle type you are searching a leak:

image

PerfView for Advanced Recording

The only other tool I know of which can enable handle leak tracing is PerfView v1.9 from 2/19/2016 or later

image

PerfView has the unique capability to stop tracing based on a performance counter threshold. This is extremely useful to find e.g. a sudden handle spike which occurs during a stress test over night at 5 a.m. in the morning but when you arrive at 6 a.m. (you are already too late Zwinkerndes Smiley) at the office the handle spike will long be overwritten by newer handle allocations of the 500MB ring buffer. Now you can get your breakfast and arrive relaxed at 9 a.m where you can start analyzing the random handle spike which your colleagues were missing while they were sitting in front of Windbg over night and present the results at 10 a.m in the morning to your manager.

The only issue I have with PerfView is that its performance counter query is locale sensitive which makes it not trivial to specify it on e.g. a Hungarian machine. For the record: On my German machine I can start Handle leak tracing which stops when the performance counter for the the first devenv instance has a value greater than 2000 handles with

  • perfview collect c:\temp\HandleLeak.etl /kernelEvents=Handle /StopOnPerfCounter:"Prozess:Handleanzahl:devenv>2000"

The feature finally seems to have been set free with the Windows 10 SDK but handle leak tracing exists also since Windows 8.1 (0?) in the kernel but no tool was capable to enable it until now. Before that ETW feature Handle leaks have been quite hard to track down but with such advanced and pretty easy to use tooling it is just a matter of two command line calls to get all allocated handles from all processes in one go.

If you leak User (Windows, Menus, Cursors, …) or GDI objects (Device Contexts, Brushes, Fonts, …) you still need to resort to intercepting the corresponding OS methods in your target process like I have shown in Generic Resource Leak Detection with ETW and EasyHook but as usual you need to use the right tool for the job at hand to nail all bugs of your application.

Conclusions

With the addition of ETW tracing to handle allocations it has never been so easy to solve handle leaks. Previously it was a pretty complex undertaking but now you can follow the steps above and you will have a nearly 100% fix rate if you analyze the gathered data correctly. If this has helped you to solve a long searched leak or you have other useful information you want to share sound off in the comments.

posted on Monday, March 14, 2016 7:09 AM

Feedback

# re: Easy Handle Leak Detection Without A Debugger 3/17/2016 3:34 AM Lucian Bargaoanu
You can also get the handle types in Process Explorer if you select Handles for the lower pane view.

# re: Easy Handle Leak Detection Without A Debugger 3/17/2016 7:50 AM Alois Kraus
Yes but ProcessExplorer cannot show a summary by handle type. You can get an aggregate with handle.exe as well via powershell:
$hash=@{}; handle | foreach {$_.Split(": ", [StringSplitOptions]::RemoveEmptyEntries)[1]} | select { $hash[$_]++ }
and then enter $hash to print the values.

Name Value
---- -----
- 1
Section 1113
v4.0 1
(C) 1
pid 107
File 2054

if you want to use Sysinternal tools only. But that will only print Section and File handles. But you can also use handle to printf a full summary with
handle -s -p devenv

Handle type summary:
<Unknown type> : 158
ALPC Port : 14
Desktop : 1
Directory : 5
Event : 513
File : 112
IoCompletion : 4
IRTimer : 6
Key : 81
Mutant : 24
Section : 81
Semaphore : 54
Thread : 103
Timer : 1
Token : 1
TpWorkerFactory : 3
WaitCompletionPacket: 21
WindowStation : 2
Total handles: 1184


Post A Comment
Title:
Name:
Email:
Comment:
Verification: