Ugh. Imagine what happens if you have a script that blocks on user input... At least on Windows XP, running the program just fails with an "Access Denied" error can lets the user handle the condition via %ERRORLEVEL%.
Why does it do this, and is there any way to tell cmd.exe to not behave this way. My original hope was perhaps there was some environment variable you could set along the lines of "NO_INTERACTIVE_MODE=1". So Let's take a closer look at what is going on under the hood by attaching a debugger to the cmd.exe while it's stuck at the dialog.
Here is the call stack taken from the MSVC debugger:
So we can see that under the hood, the command interpreter is making use of the the ShellExecute API, and that it is ShellExecute that is throwing the dialog, and not cmd.exe itself. So a quick look at the API reveals there is a flag you can send to prevent it from doing this:
http://msdn2.microsoft.com/en-us/library/bb762154(VS.85).aspx
http://msdn2.microsoft.com/en-us/library/bb759784(VS.85).aspx
If you set the SEE_MASK_FLAG_NO_UI flag in the SHELLEXECUTEINFO fMask field, errors will not cause an alert dialog to popup. Perfect.
So now we know that it's ShellExecute creating the dialog, and it can be disabled. Is there any condition under which cmd.exe will set that flag when calling ShellExecute()? Let's look at the diassembly in the debugger:
Or better yet, let's skip to two hours later when I gave up trying to analyze the above and just ran cmd.exe through the absolutely fabulous IDA Pro disassembler:
What a difference. It's moments like this when I realize how much better IDA Pro is compared to the disassembler built into Visual Studio.
From the above, it's pretty easy to see that the fMask field is hard-coded to the constant value 0x88140. So we take a look at shellapi.h in the Platform SDK, and we can see that the flag we are interested in is 0x400:
#define SEE_MASK_FLAG_NO_UI 0x00000400
So no, the flag is never set and since there is no conditional logic that dictates what gets stuck into the fMask field, its definitive that there is no way to not get the dialog box to come up if invoked through cmd.exe.
Guess I'll have to find some alternative to check the integrity of the executable before invoking it...
