Logos Bible Software Code Blog

November 5, 2009

How to Crash every WPF application

Now that we’ve released Logos 4, our users are really helping us stress-test WPF. (It’s still a little remarkable that after three years, ours is the first (and only!) WPF application installed on many of our users’ systems.)

On one system, the application was crashing at startup, with the following exception:

System.TypeInitializationException: The type initializer for
  'System.Windows.Media.FontFamily' threw an exception. --->
  System.ArgumentException: Illegal characters in path. 
    at System.IO.Path.CheckInvalidPathChars(String path) 
    at System.IO.Path.GetFileName(String path) 
    at MS.Internal.FontCache.FontSourceCollection.SetFontSources() 
    at MS.Internal.FontCache.FontSourceCollection.GetEnumerator() 
    at MS.Internal.FontCache.FamilyCollection.BuildFamilyList(List`1& familyList,
      SortedDictionary`2& familyNameList, SortedList`2& frequentStrings) 
    at MS.Internal.FontCache.FamilyCollection.MS.Internal.FontCache.
      IFontCacheElement.AddToCache(CheckedPointer newPointer, ElementCacher cacher) 
    at MS.Internal.FontCache.HashTable.Lookup(IFontCacheElement e, Boolean add) 
    at MS.Internal.FontCache.CacheManager.Lookup(IFontCacheElement e) 
    at System.Windows.Media.FontFamily.PreCreateDefaultFamilyCollection() 
    at System.Windows.Media.FontFamily..cctor()

We traced this to having an illegal path char in one of the fonts listed in the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts registry key. To reproduce the problem, you can simply edit one of those values on your own machine and add a colon, pipe, or any other illegal path character to one of the values. Now, any WPF application on your system will crash as soon as it attempts to display its first UI.

If the user of your application discovers this issue, the only thing to do is to examine each of the Fonts registry values and correct/delete any that contain invalid characters. (Or write a program to do this for you.)

I’ve filed this as Connect issue 508419; we’ve also noted that others have encountered the same problem (and fixed it in a similar way).

Posted by Bradley Grainger at 10:24 AM | Comments (2) | TrackBack (0)

November 2, 2009

Hiatus

If you’ve been following the company blog, you’ve probably guessed why this  blog has been quiet for so long. Today (after four years of development, and two months of intense private beta testing) we released Logos Bible Software 4, the major upgrade to our flagship product. This was a ground-up rewrite in C#, WPF, and .NET 3.5 (as you can probably tell from the topics previously covered here), with a dash of C++/CLI and legacy code.

We got to play with a lot of cool new technology as we developed it (we started developing with VS2008 Beta 2, and upgraded to pre-release versions of WPF along the way) and learnt a lot, which I hope we can turn into some interesting blog topics in the next few months. And now that we’re “done”, we’re looking forward to upgrading to .NET 4.0 and targeting the new features introduced in Windows 7. (And I hope our growing Mac team might chime in with some OS X-specific posts, too.)

Meanwhile, our web team delivered major improvements to the infrastructure behind logos.com (which supports Logos 4), as well as developing the back-end for our iPhone app.

I think this is the part where I mention that we’re hiring… if you’re interested in Bible software, iPhone, WPF, web, Mac, or slaving away on a 15-year-old legacy codebase (just kidding!), we’ve got a spot on our growing teams.

Posted by Bradley Grainger at 9:59 PM | Comments (1) | TrackBack (0)

September 12, 2009

"File not found" CryptographicException

The Data Protection API (in Windows 2000 or later) provides methods to securely store secret information (e.g., a cached password) on a local computer. Only the logged-in user can decrypt the protected data (if they also have the key that was used to protect it). In .NET, these APIs are exposed through the ProtectedData class.

On one of our XP test systems, a call to ProtectedData.Protect unexpectedly failed with a CryptographicException that had a very puzzling exception message:

System.Security.Cryptography.CryptographicException: The system cannot
    find the file specified. 
  at System.Security.Cryptography.ProtectedData.Protect(Byte[] userData,
    Byte[] optionalEntropy, DataProtectionScope scope)

There is no information provided on which file is required or why it's missing, or even why protecting data (in memory) requires file system access in the first place. Searching the internet for the error message turned up just a few other programmers who were also experiencing, but no solutions.

Since the ProtectedData class is just a thin wrapper around the Win32 CryptProtectData function, I searched for the underlying Win32 error code, and found the answer: the crypto methods read the HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders registry key; if there are missing values, the methods will fail. (This makes me think that the methods are incorrectly reading the "User Shell Folders" key, instead of calling SHGetFolderPath, as Raymond recommends, but I haven't been able to test that yet.)

Posted by Bradley Grainger at 1:56 PM | Comments (4) | TrackBack (0)

August 19, 2009

Entity Framework Performance Tip - Be a Minimalist

Only get the data you need! This not only applies to Entity Framework query performance, but just about every situation were code interacts with data. It is a simple rule to follow, but yet I see it broken all the time. Breaking this rule is one of the top three reasons I've seen for slow Entity Framework query performance. I may discuss the other two in later posts ...

Bad

Good

Posted by Bill Simpkins at 8:11 PM | Comments (1) | TrackBack (0)

July 30, 2009

A Few Tips For Taking a SQL Server Database Offline

In SQL Server Management Studio (SSMS), you can right-click on a database, select "Tasks" and take the database offline.

If the database seems to "hang" in the transition for more than a minute after you do this (you may be sweating bullets by this point), try the following two things:

1. Make sure you are not profiling the database.
2. Restart SSMS.

These simple things may save you a lot of stress!

Posted by Bill Simpkins at 5:29 PM | Comments (0) | TrackBack (0)

June 25, 2009

Patching a Crash in Kensington MouseWorks

A co-worker had trouble installing the latest version of an application I've been working on. The log file showed that the problem was an access violation when our setup program called MsiInstallProduct. This seemed very unusual, since Windows APIs return error codes instead of crashing. I decided to see where the crash was happening by running the installer under WinDbg.

I set WinDbg to immediately break on exceptions and launched the app. It stopped almost immediately with the following callstack. The crash location is in kwm_dll.dll, which is a hook DLL installed by Kensington MouseWorks.

kmw_dll!CallWndProcFunc+0xa8
USER32!DispatchHookA+0x101
USER32!fnHkINLPCWPSTRUCTA+0x4f
USER32!__fnDWORD+0x24
ntdll!KiUserCallbackDispatcher+0x13
USER32!NtUserSetFocus+0xc
USER32!CreateDialogIndirectParamAorW+0x33
USER32!CreateDialogParamW+0x49
msi!CBasicUI::CreateProgressDialog+0x35
msi!CBasicUI::CheckDialog+0x47
msi!CBasicUI::SetProgressData+0x58
msi!CBasicUI::Initialize+0x11c
msi!MsiUIMessageContext::Initialize+0x230
msi!MsiUIMessageContext::RunInstall+0x22
msi!RunEngine+0xe0
msi!MsiInstallProductW+0xa1
BatchUpd!Application::RunCommandInstall+0x1f5
BatchUpd!Application::Run+0x11e3
BatchUpd!wWinMain+0x102

Disassembling the crash location showed the following (crashing code in bold; ebp does not contain a valid address).

mov     eax,dword ptr [ebp-4]
mov     ecx,dword ptr [eax]
push    ecx
mov     edx,dword ptr [ebp-4]
mov     eax,dword ptr [edx+4]
push    eax
mov     ecx,dword ptr [ebp-4]
mov     edx,dword ptr [ecx+0Ch]
push    edx
call    kmw_dll!CallWndProcFunc+0x12f0 (10004210)
add     esp,0Ch
mov     eax,dword ptr [ebp+10h]
push    eax
mov     ecx,dword ptr [ebp+0Ch]
push    ecx
mov     edx,dword ptr [ebp+8]
push    edx
mov     eax,dword ptr [kmw_dll!ShowOptsProc+0x6b94 (1000e0a4)]
push    eax
call    dword ptr [kmw_dll!ShowOptsProc+0x7c80 (1000f190)]
mov     esp,ebp
pop     ebp
ret     0Ch

One interesting thing about the code is that it looks like a Debug build (or a Release build with no optimizations): the disassembly is straightforward and seems like it has a one-to-one correspondence with the putative source code. The other thing of note (not shown above) is that the base address of the DLL is set to the default 0x10000000, which is a poor choice for a hook DLL that will be loaded into every process on the system.

ebp should be preserved across the function call, so I looked at the function that was just called (at address 0x10004210). I've added a few explanatory comments based on my understanding of what it's doing.

push    ebp				; save caller's value of ebp
mov     ebp,esp				; standard function prologue
sub     esp,offset +0x87 (00000088)	; BOOL bLocal0; char szLocal1[132];
cmp     dword ptr [ebp+0Ch],0		; if (param2 == 0)
je      kmw_dll!CallWndProcFunc+0x130b (1000422b) ; goto label0;
mov     dword ptr [ebp-88h],0		; bLocal0 = FALSE;
jmp     kmw_dll!CallWndProcFunc+0x1315 (10004235) ; goto label1;
label0:
mov     dword ptr [ebp-88h],offset  (00000001) ; bLocal0 = TRUE;
label1:
mov     eax,dword ptr [ebp-88h]		; push bLocal0
push    eax
lea     ecx,[ebp-84h]			; push &szLocal1[0]
push    ecx
mov     edx,dword ptr [ebp+10h]		; push param3
push    edx
mov     eax,dword ptr [ebp+8]		; push param1
push    eax
call    kmw_dll!ShowOptsProc+0x16e0 (10008bf0)	; fn(param1, param3, szLocal1, bLocal0)
add     esp,10h				; clean up parameters (C calling convention)
mov     esp,ebp				; "free" locals
pop     ebp				; restore caller's value of ebp
ret                                                                                                  

This function is allocating 0x88 (i.e., 136) bytes for local variable storage: enough for an int (or BOOL) and a 132 byte buffer. If this buffer were overflowed, the stack would be overwritten and ebp would be corrupted upon return. Some internet searching turns up posts that discuss a similar issue, stating that "Kensington MouseWorks ... crashes ... if the executable path is longer than 128 characters"; this seems to match our situation. Indeed, dumping the bytes at the old value of ebp-84h shows the full path of our setup application, which is too long for the buffer.

Since the buffer is stack allocated, it would be trivial to change its size by editing the instructions that create and reference the local variables. At a minimum, the buffer should be capable of storing MAX_PATH characters. Because this function doesn't supply the actual buffer length to the function it calls, we can make it as long as we (reasonably) want. I decided to increase the size for storage of locals in this function to 300 bytes. In version 6.3.2.4 of kmw_dll.dll (which seems like it may be newer than the latest available version, published in February 2006), this can be accomplished by editing the following bytes in the file. These changes simply change the numbers 136, -136, and -132 (which are the three offsets used in the code above) to 300, -300, and -296.

The buffer should now be large enough to hold a file name up to MAX_PATH bytes long. With this new DLL installed in the C:\Windows\System32 folder, the setup program is able to launch the MSI and installation completes successfully.

Posted by Bradley Grainger at 7:45 PM | Comments (2) | TrackBack (0)

June 5, 2009

Using If-Modified-Since in HTTP Requests

Conditionally requesting the download of a web page only if it has been modified after a given time seems like it should be as simple as setting the IfModifiedSince property and making the request:

But of course it’s not that simple (as some others have noticed).

The designers of HttpWebRequest decided that some particular HTTP status codes would cause a WebException to be thrown. (As far as I can tell, this list is undocumented, but 304 “Not Modified” is one of them.) This is a vexing exception, because the situation is hardly exceptional. In fact, because it can only happen if IfModifiedSince is explicitly set (or if request.Headers were modified), one could argue that it’s quite expected and intentional. To avoid duplicating logic in the try block (for handling 200 “OK”) and in the catch block (for handling “304” Not Modified), I wrote a utility method that swallows any WebException thrown due to a ProtocolError (e.g., an “invalid” HTTP status code):

The code to consume this reads very similarly to the first snippet in this post; you just have to remember that normal errors (e.g., 404 “Not Found”) are reported through a valid HttpWebResponse, so its StatusCode property must be checked before acting on the response.

Posted by Bradley Grainger at 11:36 AM | Comments (3) | TrackBack (0)

June 1, 2009

Enumerable.Sum never returns null

I wrote the following code recently, and was surprised when ReSharper warned me that the condition is always true:

It’s even more surprising when you consider the following difference:

Here, sum1 is 1, but sum2 is null.

Since Sum<int?> never returns null (not even for any empty sequence, or a sequence containing all nulls), it’s odd that its return type is int?, implying that null is a possible return value. Anders explains that this return type is to keep the pattern of T Sum<T>(IEnumerable<T>) for nullable types.

But what if you want Sum to return null if the sequence contains a null? This is easy to simulate using Aggregate, as C# already propagates nulls properly when using the addition operator:

The initial value of 0 is specified to force the sum of an empty list to be zero; you could change it to default(int?) to make an empty list sum to null. A possible optimisation would be to rewrite it with a foreach loop that returns null as soon as the first null in the sequence is found.

Update: My very smart coworker points out that changing the initial aggregate value to default(int?) makes the function return null for any input. (This is probably a good reason to include a full unit test suite with every blog post…) A custom enumerator (or test of values.Any() first) could be used if returning null as the sum of an empty sequence is desired.

Posted by Bradley Grainger at 4:55 PM | Comments (0) | TrackBack (0)

May 6, 2009

WrappingStream Implementation

In a previous post, I mentioned that a certain problem could be solved by creating “an implementation of Stream that wraps another stream”. “Anonymous” asked recently, “Can you send me the code of your straightforward solution that is wrapper the class MemoryStream?”. Yes, but first I want to give another example of where such a class is useful.

The first time I used BinaryReader, I wrote code similar to the following:

Experienced users of BinaryReader will see the problem here: the BinaryReader class takes ownership of the Stream with which it’s constructed, and disposes it in Dispose. (This somewhat-important detail is only briefly mentioned in the documentation for the Close method.) Thus the last call to ReadByte fails; we have also closed the Stream even though our caller probably expects it to still be open.

While the easy answer is to simply not Close/Dispose the BinaryReader (it holds no unmanaged resources, and so nothing “bad” happens if it’s not disposed), I feel guilty every time I don’t dispose an IDisposable object. Moreover, tools like the .NET Memory Profiler have a profiling mode that lists all non-disposed IDisposable objects (to help find resource leaks); leaving this BinaryReader undisposed generates a false positive and makes it harder to find the real leaks.

The WrappingStream class can be of use in this scenario by providing an implementation of Stream that the BinaryReader can own and Dispose without affecting the real stream:

Here, at long last, is the code for WrappingStream:

You’ll note that this class isn’t sealed; that’s because it can be a useful base class for more specialised wrappers, some of which I hope to cover in future posts.

Posted by Bradley Grainger at 10:12 AM | Comments (3) | TrackBack (0)

April 28, 2009

How to use UMDH to find native memory leaks

The Debugging Tools for Windows packages come with a tool—UMDH.exe—that makes finding memory leaks in native code really pretty easy. There’s a Microsoft Knowledgebase Article that gives an overview of how to use the tool but it’s a little out of date. (It’s also very detailed, and I usually just want a summary.)

All these steps assume you have the latest Debugging Tools package installed and in your path. (Note that you need to run the tools for the same platform (x86 vs x64) as the target executable.)

Start Collecting Data

At an Administrator command prompt, run gflags.exe to start collecting stack traces for user-mode allocations:

gflags –i Program.exe +ust

Collect Snapshots

Start Program.exe running, and collect a baseline snapshot (this can be done from a regular command prompt):

umdh –pn:Program.exe –f:Dump1.txt

Perform the action that leaks memory, and collect a second snapshot:

umdh –pn:Program.exe –f:Dump2.txt

(If “Program.exe” is not a unique process name, the “-p:” command line argument can select a process by ID.)

Compare Snapshots

umdh –d Dump1.txt Dump2.txt > Diff.txt

Open Diff.txt in your favourite text editor (that can handle large files!). The memory leaks are listed in descending order of bytes leaked; each should be followed by the complete stack trace of the allocation call. Depending on the cause, this may either pinpoint the bug, or at least show a good place to set a breakpoint for debugging.

Stop Data Collection

The most important step in the whole process is to turn off the data collection for your application (once the memory leak is fixed), or else your program will run slowly while the OS kernel logs every memory allocation:

gflags –i Program.exe -ust

Posted by Bradley Grainger at 4:05 PM | Comments (0) | TrackBack (0)