API Hooking (Force OpenGL windowed)
Intro:
This tutorial is about API Hooking. For example I hooked an OpenGL program, and force it to run in windowed-mode. The hooking is done by overwriting the first bytes of the API function with a jump to the hooked function. This method isn’t perfect, but it does the trick and it’s surprisingly easy. The disadvantage of this technique is that you have to unhook the API function each time you need to use it yourself.
If you are not familiar with C++, than this tutorial is probably a little bit difficult for you.
Contents:
- Tools needed
- Used API’s
- Start
- The hook
- Finish
- References
Tools needed:
[Only registered and activated users can see links. Click Here To Register...]
I’m working with Visual C++ 2008, but since I only used Win32 API, any C++ Compiler should do the trick
Lesson 1
I used an open source program for this tutorial. For the source code and extra explanation I recommend you visit to [Only registered and activated users can see links. Click Here To Register...] or download the application from the attachment
OllyDbg
This program should be in the toolkit of every game hacker. I used it to view the imported API functions of the Lesson1.exe
Injector
I’ve got 3 different injectors from which two are the better ones.
GenTool from Deviated
Dll Injector, included in the tutorial on Planet-source-code.com. This injector uses a different kind of injection, which works on every windows platform.
Used API functions:
What does MSDN have to say about the used API functions:
Spoiler
CreateWindowExA - The CreateWindowEx function creates an overlapped, pop-up, or child window with an extended window style; otherwise, this function is identical to the CreateWindow function.
HWND CreateWindowExA(DWORD dwExStyle, LPCTSTR lpClassName, LPCTSTR lpWindowName, DWORD dwStyle, int x, int y, int nWidth, int nHeight, HWND hWndParent, HMENU hMenu, HINSTANCE hInstance, LPVOID lpParam);
ChangeDisplaySettingsA - The ChangeDisplaySettings function changes the settings of the default display device to the specified graphics mode.
LONG ChangeDisplaySettingsA (LPDEVMODE lpDevMode, DWORD dwflags);
DisableThreadLibraryCalls - Disables the DLL_THREAD_ATTACH and DLL_THREAD_DETACH notifications for the specified dynamic-link library (DLL). This can reduce the size of the working set for some applications.
BOOL WINAPI DisableThreadLibraryCalls(HMODULE hModule);
GetCurrentProcess - Retrieves a pseudo handle for the current process.
HANDLE WINAPI GetCurrentProcess(void);
(We don't need this one anymore thanks to Wiccaan for pointing out that a -1 Handle does the trick.)
GetModuleHandleA - Retrieves a module handle for the specified module. The module must have been loaded by the calling process.
HMODULE WINAPI GetModuleHandleA(LPCTSTR lpModuleName);
GetProcAddress - Retrieves the address of an exported function or variable from the specified dynamic-link library (DLL).
FARPROC WINAPI GetProcAddress(HMODULE hModule, LPCSTR lpProcName);
MessageBoxA - Displays a modal dialog box that contains a system icon, a set of buttons, and a brief application-specific message, such as status or error information. The message box returns an integer value that indicates which button the user clicked.
int MessageBoxA( HWND hWnd, LPCTSTR lpText, LPCTSTR lpCaption, UINT uType);
ReadProcessMemory - Reads data from an area of memory in a specified process. The entire area to be read must be accessible or the operation fails.
BOOL WINAPI ReadProcessMemory(HANDLE hProcess, LPCVOID lpBaseAddress, LPVOID lpBuffer, SIZE_T nSize, SIZE_T *lpNumberOfBytesRead);
WriteProcessMemory - Writes data to an area of memory in a specified process. The entire area to be written to must be accessible or the operation fails.
BOOL WINAPI WriteProcessMemory(HANDLE hProcess, LPVOID lpBaseAddress, LPCVOID lpBuffer, SIZE_T nSize, SIZE_T *lpNumberOfBytesWritten);
ShowCursor - The ShowCursor function displays or hides the cursor.
int ShowCursor(BOOL bShow);
Start:
- Now it’s time for the real work. Fire up OllyDbg and open “Lesson1.exe”. (Click on File->Open (or just hit F3 on your keyboard), browse to the directory that contains Lesson1.exe and click on Open.)
- Next click on the Blue “E” in the top of the screen, you will see a list of all loaded Executable Modules in the process.
- Right-Click on the one that has Lesson1.exe in the end of his name.
- Click on “View Names”.
- What you see now is a list of all API functions imported by Lesson1.exe.
If you go through the list you’ll see CreateWindowExA and ChangeDisplaySettingsA. Those are the two API’s we’ve been looking for. If you look at the bottom of the list you will see another API function, ShowCursor. We are going to need that one too.
So now we have three API functions we need to deal with. CreateWindowExA, ChangeDisplaySettingsA and ShowCursor. Search MSDN for these API functions. It’s important that you know what these functions do. This once I’ve posted the info in the section above. Read it if you haven’t done this jet. Let’s get on to the next chapter.
The Hook:
Start Visual C++, or whatever compiler you are using. But remember this tutorial will be describing it from the Visual C++ point of view.
Set up][*]Start a new Project: Click on File->New->Project (or press Ctrl+Shift+N).[*]As ProjectType select “Win32”.[*]As Template select “Win32 Project”.[*]Put a name into the box that says Name (I use “GLHook”). Make sure that “Create Directory for solution” is Checked and click on “Ok".
[*]You will see some kind of wizard, click “Next”, by Application Type select “Dll”. Make sure to check the “Empty Project” checkbox.
[*]First things first. We need to set the right properties for this project. Go to Project->GLHook Properties… (or hit Alt+F7).
Go to “Configuration Manager”. Click on “Debug” and Select “Release”, now you can close the Configuration Manager.
Click “Configuration Properties”, “General”.
Change “Character Set” To “Use Multi-Byte Character Set”.
Change “Whole program Optimization” to “No Whole Program Optimization”.
Click on “Apply”.
[*]To the left you will see the “Solution Explorer”. Right-Click on the Directory that is called “Header Files”. Click on Add->New Item, Then select “Header File (.h)”. Type a name(“GLHook”) in the name box and hit “Ok”.
[*]Do the same think for the directory that’s called “Source Files”, but now select “C++ File (.cpp)” Give it a name (“GLHook”) and click “Add”.[*]Do the same once more, but now name it (“Main”).[/list]
Programming:
Let’s start with the easiest file of all. Open “GLHook.h”.
Add the following line of code:
DWORD HookGeneralFunction(const char *Dll, const char *FuncName, void *Function, unsigned char *backup);
This piece of code is called a function prototype. It tells the compiler what kind of variables this function needs as input, and what kind of variable it returns.
HookGeneralFunction is the function name. The variable it returns is a DWORD. This function takes 4 variables as input. This file is done. Save and close it.
Next open the “GLHook.cpp” file. We are going to program the HookGeneralFunction function.
First we include the necessary headers:
“Windows.h” contains all the function declarations of the API, “Stdio.h” contains the standard Input/Output functions.
#include <windows.h>
#include <stdio.h>
Next we define the HookGeneralFunction function:
DWORD HookGeneralFunction(const char *Dll, const char *FuncName, void *Function,
unsigned char *backup)
This function takes 4 parameters as input.
Parameters Are][*]Dll - Name of the Dll that contains the API function.[*]FuncName - Name of the API function you want to hook.[*]Function - Name of the function the API function gets redirected to.[*]backup - Array of bytes the original code will be read to.[/list]
Next line of code:
DWORD addr = (DWORD)GetProcAddress(GetModuleHandle(Dll), FuncName);
What does this code do? First we get a handle to the Dll that contains the API function we want to hook using GetModuleHandle. This API function takes the Dll name as input.
Next we get the address of the API function we want to hook, by using GetProcAddress. This API function takes a Module handle and a function name as input.
The result(the address of the API function we want to hook) is stored in a DWORD that is called “addr”.
Ok, let’s continue:
BYTE jmp[6] = {0xE9,0x00, 0x00, 0x00, 0x00,0xC3};
ReadProcessMemory((HANDLE)-1, (void*)addr, backup, 6, 0);
We create an array of Bytes(which is called jmp) and set the first byte to 0xE9(ASM code for long jump) then we skip 4 byte(these 4 bytes will contain the address of the function the API function gets redirected to) and then we set the last byte to 0xC3(which is ASM for RETurn).
The second line of code uses the ReadProcessMemory API function to read the first 6 bytes of the API function we want to hook, and stores them into Array of Bytes we got as input.
DWORD calc = ((DWORD)Function - addr - 5);
memcpy(&jmp[1], &calc, 4);
The first line of code calculates the jump. The second line of code puts the calculated jump address into the array of Bytes called jmp(which we created earlier).
WriteProcessMemory((HANDLE) -1, (void*)addr, jmp, 6, 0);
return addr;
}
The first line of code uses the WriteProcessMemory API function to write the array that contains the jump code, into the API function we want to hook.
The second line returns the address of the hooked API function to the caller function.
This was all the coding for this file. Save and close it.
Alright up to the last file, open “Main.cpp”. Let’s start with the usual includes.
#include <windows.h>
#include <stdio.h>
#include “GLHook.h”
The next thing to do: add the function prototypes.
LONG WINAPI hook_ChangeDisplaySettings(LPDEVMODE lpDevMode, DWORD dwFlags);
HWND WINAPI hook_CreateWindowEx(DWORD dwExStyle, LPCTSTR lpClassName, LPCTSTR lpWindowName, DWORD dwStyle, int x, int y,int nWidth, int nHeight, HWND hWndParent, HMENU hMenu, HINSTANCE hInstance, LPVOID lpParam);
After that, we declare some Global variables:
DWORD CreateWindowExaddr=0;
DWORD ChangeDisplaySettingsaddr=0;
BYTE backupCW[6];
BYTE backupCDS[6];
We create 2 DWORDs to hold the addresses returned by the HookGeneralFunction function, when we hook the API functions.
We also create 2 Arrays of Bytes to store the original code of the API functions we are going to hook (in case we need to use the API functions ourselves, we need to restore the original API function, or else we end up in our own hook function).
Next we define the Dll’s main function:
BOOL APIENTRY DllMain(HINSTANCE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
This function gets called every time a message is sent to our dll. The messages we are waiting for are DLL_PROCESS_ATTACH (when the dll is just loaded) and DLL_PROCESS_DETACH (when the dll gets unloaded, usually when the process is terminated).
If you have ever programmed in C++ before, you know what we are going to do next. We are creating a switch to handle “ul_reason_for_call”.
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
The code in “case DLL_PROCESS_ATTACH:” gets executed when the dll is first loaded.
DisableThreadLibraryCalls(hModule);
MessageBox(0, "Attached!", "Dll Attach", 0);
The first line of code disables the DLL_THREAD_ATTACH and DLL_THREAD_DETACH notifications for our dll.
The second line creates a message box that tells us that the dll is attached.
CreateWindowExaddr = HookGeneralFunction("user32.dll", "CreateWindowExA", hook_CreateWindowEx, backupCW);
ChangeDisplaySettingsaddr = HookGeneralFunction("user32.dll", "ChangeDisplaySettingsA", hook_ChangeDisplaySettings, backupCDS);
break;
The first line of code calls HookGeneralFunction to redirect the “CreateWindowExA” API function, from the user32.dll, to our “hook_CreateWindowEx” function. The overwritten code will be saved in “backupCW”, and the API function address will be stored in “CreateWindowExaddr”.
The second line of code redirects the ChangeDisplaySettingsA function to our hook function.
The last line of code stops the switch. Since we already found the corresponding “ul_reason_for_call” there is no need to check the other cases.
case DLL_PROCESS_DETACH:
if (CreateWindowExaddr)
WriteProcessMemory((HANDLE) -1, (void*)CreateWindowExaddr, backupCW, 6, 0);
if (ChangeDisplaySettingsaddr)
WriteProcessMemory((HANDLE) -1, (void*)ChangeDisplaySettingsaddr, backupCDS, 6, 0);
MessageBox(0, "Dll Detached", "Process End", 0);
}
return TRUE;
}
We will be using “DLL_PROCESS_DETACH”, because we want to unhook the process before our dll is detached. Because the two If statements actually do the same, I will only explain one of them.
The first if statement first checks if “CreateWindowExaddr” ever has been used (if it hasn’t been used, then we don’t need to restore the hook. In fact, we would screw up the API function because we would write six 00 bytes into it). But if “CreateWindowExaddr” has been used, it means that the API function was hooked, so we unhook it by writing back the original bytes.
When the dll is detached, we will see a message box that tells us that the dll is detached.
This is the end of the “DllMain” function, so if everything went alright, we return true.
Now it’s time to set up our hookfunctions.
HWND WINAPI hook_CreateWindowEx(DWORD dwExStyle, LPCTSTR lpClassName, LPCTSTR lpWindowName, DWORD dwStyle, int x, int y, int nWidth, int nHeight, HWND hWndParent, HMENU hMenu, HINSTANCE hInstance, LPVOID lpParam)
{
This will be our function where every call made to the original “CreateWindowExA” API function will be redirected to.
If you have done your homework, you will see that the interface of our hookfunction is actually the same as the interface of the original “CreateWindowExA” API function. You need to do this for every function you want to hook ever, if you don’t, you will screw up the stack and make the program crash!
dwExStyle = WS_EX_APPWINDOW | WS_EX_WINDOWEDGE;
dwStyle = WS_OVERLAPPEDWINDOW;
lpWindowName = "Hooked!";
nWidth = 500;
nHeight = 700;
ShowCursor(TRUE);
Because we want the program to run in windowed mode, we need to change some of the variables the program passes to “CreateWindowExA”.
The first and second line of code changes the variables that hold the style of the window. We set these variables to the normal window variables.
The third line of code changes the original window name into “Hooked!”. Just for fun :P
The fourth and Fifth line set the new width and height values, we need to change them because we don’t want a full screen window.
The last line of code makes sure that the mouse pointer will be visible.
WriteProcessMemory((HANDLE) -1, (void*)CreateWindowExaddr, backupCW, 6, 0);
Because we need to use the “CreateWindowExA” function ourselves, we first need to unhook it (if we don’t we’ll end up back in our own hook function, if this happens we will be running around in circles which will end up in a crash).
HWND ret = CreateWindowEx(dwExStyle, lpClassName, lpWindowName, dwStyle | WS_CLIPSIBLINGS | WS_CLIPCHILDREN, x, y, nWidth, nHeight, hWndParent, hMenu, hInstance,lpParam);
Now we call the original “CreateWindowExA” API function, with the changed variables. The return value (a handle to the newly created window) will be stored in the “ret” variable, so that we can pass it back to the caller function.
CreateWindowExaddr = HookGeneralFunction("user32.dll", "CreateWindowExA", hook_CreateWindowEx, backupCW);
return ret;
}
The first line reinstalls the hook.
The second line returns the return value we’ve got from the original “CreateWindowExA” function.
LONG WINAPI hook_ChangeDisplaySettings(LPDEVMODE lpDevMode, DWORD dwFlags)
{
return 0;
}
The “ChangeDisplaySettingsA” API function screws up your desktop when called, so the only thing we do is return DISP_CHANGE_SUCCESFUL value (which happens to be 0) so the caller function thinks everything went alright.
All right, this concludes the programming of our hook function. To build the dll, click on Build->Build Solution, or press the F7 key.
Let’s see what happens when we inject the Dll into Lesson1.exe.
Finish:
- I will use GenTool for this tutorial, because it is user-friendly. Fire up GenTool. (If you are using Windows Vista, you will get two messages that will state that this program can be a threat, just ignore them. This program is clean, it’s just that the way it injects dll’s is used by many malicious software).
- Click the “Load Dll” button and go to our newly created dll.
- Start Lesson1.exe.
- In GenTool, click the “Refresh” button.
- Click in the listbox, and type “les” the listbox will automatically go to “Lesson1.exe”.
- Hit the “Inject Dll” button.
- If everything went OK, you will see a message box which says “Attached!” (remember “DLL_PROCESS_ATTACH”?!). Click Ok.
- There will be another message box (it showed up when you started Lesson1.exe), click yes to run in full screen mode.
- A window will appear which is titled “Hooked!”. Congratulations, Mission Accomplished.
If you know how to handle OllyDbg and you want to get a better insight into how this hook works, add the following code on top of “GLHook.cpp”:
void NumOut(char *format, ...)
{
char szBuffer[4096];
ZeroMemory(&szBuffer, sizeof(szBuffer));
va_list pArgs;
va_start(pArgs, format);
vsprintf(szBuffer, format, pArgs);
va_end(pArgs);
MessageBox(0, szBuffer, "NumOut:", 0);
}
And make the following changes in “HookGeneralFunction”
WriteProcessMemory((HANDLE) -1, (void*)addr, jmp, 6, 0);
NumOut("%x", addr); //<<<<Add this line<<<
return addr;
Now you will get the address of the API function when it gets hooked. Start OllyDbg and Attach to Lesson1.exe. Go to the given address and you will see the jump to our hook function with a return below it.
The result:
I forgot to mention before. The OllyDbg thing works only if the API function is in the programs IAT (import address table). If the dll’s are loaded using the LoadLibrary Function, you won't be able to find them this way.
The End.
Author: Tenebricosus (doxcoding.com)
Correction: Genz (progamercity.net)
You may use this anywhere, as long as you provide a link to these forums.
References:
First of all the source code that started all [Only registered and activated users can see links. Click Here To Register...].
For a better understanding of OpenGL I recommend you visit [Only registered and activated users can see links. Click Here To Register...]
If you are looking for API function interfaces, I suggest you search [Only registered and activated users can see links. Click Here To Register...]
Full source code:
[Only registered and activated users can see links. Click Here To Register...]
