Some DirectX APIs are not designed to handle high memory conditions and will report far less memory than is truly available. This behavior is described in this Microsoft knowledge base article:
Some DirectX applications will use these APIs to determine how much video memory is available and adjust their behavior (such as texture quality). Other DirectX applications, and all OpenGL applications, are unaffected.
On Windows Vista and Windows 7, NVIDIA provides GPU driver support to work-around this issue for GPUs with less than 3GB of memory. The workaround forces the video memory to the maximum amount these DirectX APIs can report, sacrificing some of the total memory. This is called "Compatibility Mode".
The application "NvTagToggle.exe" linked below can help you detect this scenario and configure this workaround, as well as reset the GPU driver to its default behavior. The application is only supported on Windows Vista and Windows 7, for NVIDIA GPUs with less than 3GB of memory.