- Contents
- Introduction
- Humble Beginnings
- The Guts
- A Quick test
- Wrap Up
- Appendix: Other Interesting Behaviour
Introduction
Recently WinDevs, the guys in charge of developing the Getting Started content for native developers on MSDN, and I had a short conversation relating to message loops. They’re in the process of creating a Win32 tutorial in the vein of TheForgers teaching the basics of creating a small Direct2D drawing app. The message loop used in their tutorials is the ubiquitous:
MSG msg; while(GetMessage(&msg, NULL, 0, 0)) { TranslateMessage(&msg); DispatchMessage(&msg); }
On a forum I visit, one of the denizens had pointed out in response to an unrelated query that the GetMessage documentation states -1 can be returned on failure and that the loop above should not be used, instead advising programmers to use:
MSG msg; BOOL bRet; while( (bRet = GetMessage( &msg, NULL, 0, 0 )) != 0) { if (bRet == -1) { // handle the error and possibly exit } else { TranslateMessage(&msg); DispatchMessage(&msg); } }
As the tutorial is aimed at beginners and no doubt intended to teach best practices, I mentioned the contradictory advice to them. After a bit of investigation, they’ve since explained the incongruity. MSDN mentions as examples that an invalid HWND and an invalid message pointer can be causes of a -1 return. Being curious about such things, I wondered what exactly causes such a return.
Humble Beginnings
In the grand scheme of things, GetMessage is a simple API. Firstly, it validates some input:
; ; GetMessageW, user32.dll, Win7 ; mov edx, [ebp+wMsgFilterMin] mov ecx, [ebp+wMsgFilterMax] or edx, ecx ; or together both passed in filter values mov eax, 0FFFE0000h test edx, eax ; and them with 0xFFFE0000 to check if any of the high 15 bits are set jnz loc_77D4CF3A ; if any are, go to the following section ; --- ; loc_77D4CF3A ; --- cmp ecx, 0FFFFFFFFh ; Compare the max filter with UINT_MAX jnz short loc_77D4CF4B ; If it's not that, go to the location below test [ebp+wMsgFilterMin], eax jnz short loc_77D4CF4B ; if the min filter value isn't 0xFFFE0000, go to below location xor ecx, ecx ; otherwise set max filter to 0 jmp loc_77D28FB1 ; and go call into the kernel ; --- ; loc_77D4CF4B ; --- push 57h call _UserSetLastError@4 ; SetLastError(Error_Invalid_parameter) xor eax, eax ; set return value as false jmp loc_77D28FDE ; goto following code ; --- ; loc_77D28FDE ; --- pop ebp retn 10h ; return from function
Straightaway, we can see that passing an invalid message filter range won’t cause a -1 return, so that’s one off the list. After this validatation, the next upheaval is the transition to kernel mode, via a call to NtUserGetMessage.
The Guts
Upon entry to NtUserGetMessage, the message filters are validated again in the same way as above (because the function can be called direct in kernel mode) before a flying visit to xxxInternalGetMessage dumps us into xxxRealInternalGetMessage which is the workhorse for both Peek and GetMessage.
Three functions on from where we started, and the HWND value is finally examined:
; ; xxxRealInternalGetMessage, win32k.sys ; push 24h push offset unk_BF9FF400 call __SEH_prolog4 ; Setup SEH, we'll see this later mov esi, _gptiCurrentThreadInfo mov ebx, [ebp+hwnd] cmp ebx, 0FFFFFFFFh ; if the passed in HWND is (HWND)-1... jz short loc_BF8B95F6 cmp ebx, 0FFFFh jnz short loc_BF8B95f9 ; or (HWND)0xFFFF... ; --- ; loc_BF8B95F6 ; --- xor ebx, ebx inc ebx ; set the HWND to 1 ; --- ; loc_BF8B95F9 ; --- test ebx, ebx jz short loc_BF8B9645 ; if HWND is NULL, go do thread message work cmp ebx, 1 jz short loc_BF8B9645 ; likewise if it's 1 mov ecx, ebx call @ValidateHwnd@4 ; Otherwise check it's a valid window mov edi, eax mov [ebp+var_1C], edi ; Save the HWND's associated PWND struct on the stack test edi, edi jnz short loc_BF8B9626 ; check the PWND is not NULL. If it isn't, do window work mov eax, [ebp+pMsg] and [eax], edi ; otherwise, set MSG.hwnd to NULL and [eax+4], edi ; and MSG.message to WM_NULL ; now here's the clever part. PeekMessage returns 0 on invalid HWND ; but GetMessage returns -1, this does both depending on the value of fromGetMessage mov eax, [ebp+fromGetMessage] neg eax sbb eax, eax jmp loc_BF8B9B78 ; go to exit ; --- ; loc_BF8B9B78 ; --- call __SEH_epilog4 ; teardown the SEH frame retn 18h ; return and pop the stack
The documentation turns out to be correct and we have a -1 return when passed an invalid HWND. The other special window constants of -1 and 0xFFFF are documented values for PeekMessage to take but by virtue of the shared code are valid for GetMessage too.
As there’s an active SEH frame, let’s see what happens when an exception happens:
; --- ; loc_BF8B9B5E, the exception filter ; This is an equivalent of ; __except(__W32ExceptionHandler(GetExceptionCode())) ; --- mov eax, [ebp-14h] mov eax, [eax] push dword ptr [eax] call __W32ExceptionHandler@4 ; always returns EXCEPTION_EXECUTE_HANDLER retn ; which leads into... ; --- ; loc_BF8B9B6B, the exception handler body ; --- mov esp, [ebp-18h] ; reset stack pointer to the functions stack space mov dword ptr [ebp-4], 0FFFFFFFEh or eax, 0FFFFFFFFh ; set return value to -1 ; --- ; loc_BF8B9B78 ; --- call __SEH_epilog4 ; teardown the SEH frame retn 18h ; return -1
So, a second way for the function to return -1 is if an exception happens during its execution. This part only applies to Vista onwards, on XP there is no such exception handler set up in the function. If one occurs, a handler created by NtUserGetMessage will be entered, returning 0.
For non-exceptional cases other than the window validation, there’s only one exit route out of the function. In the second half, the ebx register contains the return value. It’s only assigned to once, after a message has been picked:
cmp [ebp+fromGetMessage], 0 ; if from PeekMessage jz short loc_BF8B99C5 ; set return = 1 cmp dword ptr [edi+4], 12h ; otherwise if from GetMessage and msg == WM_QUIT jz loc_BF8B9A8E ; set return to 0, otherwise continue and set return to 1 ; --- ; loc_BF8B99C5 ; --- xor ebx, ebx inc ebx ; --- ; loc_BF8B9A8E ; --- xor ebx, ebx
As far as the return value goes, that’s that. On Windows 7, we have four exit routes with a possible three values and two occasions we can receive -1.
A Quick Test
Windows 7 represents only one of the OS’s MSDN currently supports and while you can look at assembly all day, it’s resource intensive and not exactly fun. The simplest way to find out what happens on the other OS’s is to just try it. So that’s what I did. Note 1 contains some code that performs the five failure tests over which the user has control. After running it on various VM’s I had laying around, here are the results. LE denotes the return from GetLastError.
98 SE, this test used GetMessageA instead of GetMessageW like the others:
High word flags returned 1, (LE = 0) NULL Msg returned 1, (LE = 0) Read-Only Msg returned 1, (LE = 0) Invalid Msg returned 1, (LE = 0) Invalid HWND returned ffffffff, (LE = 0)
NT4 SP2:
High word flags returned 0, (LE = 87) NULL Msg returned 0, (LE = 0) Read-Only Msg returned 0, (LE = 0) Invalid Msg returned 0, (LE = 0) Invalid HWND returned ffffffff, (LE = 1400)
XP SP3:
High word flags returned 0, (LE = 87) NULL Msg EXCEPTION returned c0000005, (LE = 998) Read-Only Msg returned 0, (LE = 998) Invalid Msg EXCEPTION returned c0000005, (LE = 998) Invalid HWND returned ffffffff, (LE = 1400)
7 SP0:
High word flags returned 0, (LE = 87) NULL Msg EXCEPTION returned c0000005, (LE = 998) Read-Only Msg returned 0, (LE = 998) Invalid Msg EXCEPTION returned c0000005, (LE = 998) Invalid HWND returned ffffffff, (LE = 1400)
Errors:
87 = ERROR_INVALID_PARAMETER
998 = ERROR_NO_ACCESS
1400 = ERROR_INVALID_WINDOW_HANDLE
A set of somewhat suprising results. It seems only NT4 had sane behaviour, with 98 not giving a hoot and returning success regardless. One explanation for this is that it returned whether there were messages present when given incorrect parameters, but that’s just a guess.
Slightly more worrying are the exceptions produced by XP and 7. This pours water on the claims that an invalid MSG pointer will return -1, when in such situations it may not return in the conventional sense at all. Let’s not forget common sense however, is anybody going to be calling a function whose sole operation is to fill in a struct and give it a bad pointer?
Wrap Up
Knowning what we know now, we can see that the return value section of the GetMessage doc page is correct, if only partially. Out of the 5 failure cases over which a user has control, only an invalid, non-null HWND actually returns -1. An invalid message pointer never does, returning 0 on older NT and 1 on 9x Windows, but worse, causing an access violation on new when given a completely wild pointer.
One full circle later and where have we ended up? Barring internal errors (which I’ve never encountered) GetMessage(&msg, NULL, 0, 0) will only return 0 upon receiving WM_QUIT with -1 an impossibility. In regards to Win devs and their message loops everwhere, it’s as you were.
Other Interesting Behaviour
Lost Messages
Something glossed over earlier in the transition between NtUserGetMessage and xxxGetMessage, is that a second MSG structure allocated on the kernel stack is actually filled in first rather than the pointer you give to GetMessage. After xxxGetMessage completes (successfully or not), its contents are copied to the original MSG structure. If something goes wrong during this copy, the message is lost. There are no second chances.
The Exceptions
XP and 7 cause exceptions when given a bad message because GetMessage doesn’t check the return value of NtUserGetMessage at all, nor the validity of the message pointer before dereferencing it.
push esi mov esi,dword ptr [ebp+8] push ecx ; wFilterMax push dword ptr [ebp+10h] ; wFilterMin push dword ptr [ebp+0Ch] ; hWnd push esi ; pMsg call _NtUserGetMessage@16 mov ecx,dword ptr [esi+4] ; access pMsg->message, if it's bad it goes boom cmp ecx,102h ; check if its WM_CHAR... je _GetMessageW@16+46h cmp ecx,0CCh ; or EM_SETPASSWORDCHAR je _GetMessageW@16+46h pop esi pop ebp ret 10h
Note 1: The test program
// compile flags // cl /GS- /DUNICODE /D_UNICODE /Ox test.cpp /link user32.lib kernel32.lib /subsystem:console /entry:entryPoint #define WIN32_LEAN_AND_MEAN #include <windows.h> HANDLE g_hstdOut = NULL; void PrintResult(LPCTSTR desc, BOOL ret) { TCHAR buffer[150]; int written = wsprintf(buffer, TEXT("%s returned %x, (LE = %d)\r\n"), desc, ret, GetLastError()); DWORD wrote; WriteFile(g_hstdOut, buffer, written * sizeof(buffer[0]), &wrote, NULL); } #define DO_TEST(msg, hwnd, min, max, desc) \ do \ { \ __try \ { \ SetLastError(0); \ BOOL ret = GetMessage((msg), (hwnd), (min), (max)); \ PrintResult((desc), ret); \ } \ __except(EXCEPTION_EXECUTE_HANDLER) \ { \ PrintResult(desc TEXT(" EXCEPTION"), GetExceptionCode()); \ } \ } \ while(0) static const MSG readOnlyMessage = {0}; extern "C" int WINAPI entryPoint() { g_hstdOut = GetStdHandle(STD_OUTPUT_HANDLE); MSG msg = {0}; // create the message queue PeekMessage(&msg, NULL, 0, 0, PM_NOREMOVE); // make sure we have a message so GM won't block if it succeeds for(int i = 0; i < 10; ++i) PostThreadMessage(GetCurrentThreadId(), WM_APP, 0, 0); // high word wake flags don't see to be taken into account pre Win2000 DO_TEST(&msg, NULL, 0x12345678, 0xDCBA9876, TEXT("High word flags")); DO_TEST(NULL, NULL, 0, 0, TEXT("NULL Msg")); DO_TEST((PMSG)&readOnlyMessage, NULL, 0, 0, TEXT("Read-Only Msg")); DO_TEST((PMSG)3, NULL, 0, 0, TEXT("Invalid Msg")); DO_TEST(&msg, (HWND)0x9, 0, 0, TEXT("Invalid HWND")); ExitProcess(0); }