Process Monitor : Is there a way of hooking GlobalAlloc/Free()?

Hooking GlobalAlloc and GlobalFree won't do you any good. These functions are officially obsolete and most processes don't use them much, if at all. They are only maintained for backward compatibility and for those few purposes where they are still needed.

To test for a memory leak you would need to monitor the commit contribution of processes in Task Manager. In XP this will be called "VM Size" and in Vista and later "Commit size". You will need to add these columns as they are not there by default. If a process has a memory leak it's commit charge will rise over time, usually to a high level. Checking physical memory usage won't be very useful.