Microsoft Code Analysis results differ based on Configuration

Background

Code Analysis (CA) is a very useful Visual Studio feature that applies good practice type static code analysis to your code base.  You can run CA manually at any time or configure it to run automatically on each build. Automatic execution can be configured per configuration per project. This means you can enable automatic CA on Release builds while ignoring it during builds in Debug mode.  Some teams do this to speed up the debug compilation cycle.  I'm not sure what in CA makes it so slow that you can't run it all the time :-(

You can view which CA rules apply for any configuration or CPU type via the Solution Properties window:

You can set the automatic execution of CA on a per project basis in the Code Analysis pane of the Project Properties:

The method of Code Analysis  menu item and the position of the results varies by Visual Studio Version. The menu item that executes CA is located on the Build menu in VS2013.

Unexpected Behavior

Microsoft Code Analysis will sometimes generate different results based on the configuration.  This may be the result of rule configuration, flag values and/or the level of optimization. I've used static analysis tools on Java for years but this is the first time I've seen different results.  That may be because we only compiled Java one way.

This very simple method generates different results with Microsoft All Rules depending on the configuration.

        public void DummyMethod(){
            HttpStyleUriParser foo = new HttpStyleUriParser();
        }

You can see here that the last Code Analysis error is different. Make sure you are testing your code with CA in the proper configuration for your project's standards.

Release Configuration	Debug Configuration

We found this when using nested C# using blocks to insure the correct clean up behavior in stream and memory buffer operations.  A CA rule fires in Debug configuration that flags the way using blocks map to close and Dispose() behavor. This Code Analysis error disappears in Release configuration , possibly because the system intelligently calculates the correct behavior.

Conclusion

Make sure you run your CA rules in the same configuration as your build servers.  You don't want to think you are clean and then have the CI builds fail because of a CA related issue.