I don't really see why havinfg the dll as 'release' and the app as 'debug' should cause this problem. After all the windows api dll's your app uses would be the release versions even for a debug app.
There is a major (and not always realised) difference between release and debug builds. A debug build has variables set to an initial value and check bytes around variables. A release build just uses whatever value was in that memory block (not necessarily zero) and doesn't have the safety bytes - so a buffer overrun can result in corruption of other variables that isn't seen in a debug build.
eg.
int i;
if(i != 0)
CrashTheApp(); <<-- this shouldn't be called in debug but can occasionally be called in release.
So:
Is the release build always failing in the same function ?
In a debug build do you see reported problems in the output window of visual studio when your app exits ?