Jenkins CI in MVC.NET Part 3: Analysis and Report

In this article we will show how to setup Jenkins CI Job that run code analyze tools like StyleCop and FxCop for general .NET coding practices, CPD for code duplication and JSHint for JavaScript coding practices. This project analyze tools help for the estimate of the overall project quality, security and portability. Help estimate the effort needed to put a project back on track or see of the project is started following a bad practices in early stage before going too far off tack. This can save company money and developers headache.
This is part of the series for Jenkins CI with the previous posts here and here, but code analyze tools can be run separate in a dedicated job if you wont to.
The setup project that can be used to test this setup can be found here.

The tools

I will provide a short description of the tools used for analyze and report tasks.

• FxCop – Microsoft tool that evaluate build assemblies for some coding practices defined by a rulesets.
• StyleCop – This tool uses predefined sets of rules to evaluate the styling of the code documents and identify bad practices.
• StyleCop+ – Add some extra rules to StyleCop.
• JSHint – Fork of the JSLint framework that analyze JS files for some bad client script practises (rules can be predefined).
• CPD – Java tool that can analyze code fails for CPD (copy-paste detection). The goal is to fallow the do-not-repeat-yourself principal.
• JRE – Java runtime environment is used to run the PDM – CPD plugin. Jenkins CI include JRE, also.
• PhantomJS – Headless browser to run the JSHint against.
• Violations plugin – Jenkins CI plugin for visualizing analysis information.
• Email ext – Extended email notifications

Setup a project in Visual Studio

StyleCop

StyleCop is run as a separate target in any Visual Studio project. In order to run a code analysis in Visual Studio you need to do the following steps:

1. Download the StyleCop and StyleCop+ libraries. You can do this in two ways.
   • Download from the official sites the StyleCop and StyleCop+ and copy the files in single directory.
   • Download them from here.
2. Copy the StyleCop folder (with StyleCop and StyleCop+ files) to your project folder at “MY_SOLUTION_FOLDER/Tools/StyleCop”.
3. Edit the StyleCop.ruleset to fit your project preferences.
4. Edit all your solution .csproject to include the following import:

   <Import Project="..ToolsStyleCopStyleCop.Targets" />

5. Open Visual Studio and build to see the result.

Note: Running 2 instances of Visual Studio with different StyleCop rulesets can mixes the evaluation results.

FxCop

Visual Studio 2013 and above has integrated solution for running FxCop rules. You can find it in the Analyze menu.
In order to setup a FxCop plan you need to go to ‘Analyze -> Configure code analysis for solution’. There you can select from a Microsoft provided list of rule sets.
Generally you would like to setup your own set of rules and use that. The way you do this is as follow:

• Copying some of the Microsoft ruleset (possible location is “root:Program Files (x86)Microsoft Visual Studio XX.XTeam ToolsStatic Analysis ToolsRule Sets”) and modifying it or download a predefined rulesets like this one.
• Open the .ruleset file with text editor and change the name of the ruleset <RuleSet Name=”{MyRuleSetName}” …>.
• Place it in “MY_SOLUTION_FOLDERTools” and open it in the same Visual Studio instance your project is opened in.
• Go to ‘Analyze -> Configure code analysis for solution’ and the your ruleset should be part of the rule sets list.
• Select your rule set and run it by selecting ‘Analyze -> Run code analysis on solution’.

JSHint

You can run JSHint in Visual Studio by using the Web Essentials Visual Studio extension or by using SharpLinter and the instructions here.
Another option is to use post build script that that run it in PhantomJs as shown below.

Run Jenkins CI job

Build Variables

The build variables are the same as the ones in the previous article Jenkins CI in MVC.NET Part 2: Tests and Coverage. An extra variable is:

• CPDMAXTOKENS – the maximum number of letters that will not fire CPD violation.

StyleCop

If you have followed the instructions in setup StyleCop in Visual Studio your project have already StyleCop integrated. All you need to do is build the project with MSBuild as described here and StyleCop analyze report will be created.

Setup the report file location

In order to set the report file location you need to open the StyleCop.targets file at ‘MY_SOLUTION_FOLDERToolsStyleCopStyleCop.targets’. There you will find the following section:

  <PropertyGroup Condition="'$(StyleCopOutputFile)' == ''">
    <StyleCopOutputFile>$(SolutionDir)Reports$(ProjectName).StyleCopViolations.xml</StyleCopOutputFile>
  </PropertyGroup>

This setup will output a StyleCop violations report in the Reports folder under the WORKSPACE (Solution) folder. If your project is called MyProject.Web then the path will be ‘%WORKSPACE%ReportsMyProject.Web.StyleCopViolations.xml’.

FxCop

In order to create a FxCop analyze report file we need the FxCop command line tool. This can be downloaded from the Microsoft download page here or here (for both options you need to install Microsoft Windows SDK for Windows 7 and .NET Framework 4) or if you have Visual Studio installed use if from there (located in the ‘Team ToolsStatic Analysis Tools’ folder).
If we assume you have FxCopCmd installed in “C:ToolsFxCop” folder we can run the analysis like this:

REM 'Start FxCop to check for general coding problems.'
"C:ToolsFxCopFxCopCmd.exe" /out:ReportsFxCopViolations.xml /ruleset:=ToolsDotNetCodingStandards.ruleset /file:%WEBPROJECT%/bin/%ASSEMBLIES%.*.dll /gac /ignoregeneratedcode

After this a FxCop violations report will be generated in the Reports folder.
Note: The script assum the FxCop .ruleset file is called DotNetCodingStandards.ruleset. You must change it or name the .ruleset file like this.

JSHint

In order to run JSHint we will use PhantomJS. I have tried to use all kind of solutions to run JSHint in Jenkins and have found it difficult to do this with standard tools. So what we will do is use the PhantomJS API to run JSHint directly and output the result in xml report file. For this purpose I have written a JavaScript file that uses the phantomJs API to load the JSHint.js and the js files in a given folder recursively and check them against a set of rules.
You can find all that here. Just download it and place it inside “C:ToolsJSHint” folder and run it like this:

REM 'Start JSHint to check for JavaScripts coding problems.'
"C:ToolsJSHintphantomjs.exe" C:ToolsJSHintjshint-runner.js %WEBPROJECT%Scripts ToolsJSHintRules.json ReportsJSHintViolations.xml

• The first argument is the jshint script to perform the JSHint check.
• The second argument is path where all script files are located in (recursive check).
• The third argument is the config file. In it should be all JSHint rules like ‘{ “unused” : true, “latedef” : false, “strict” : true, … }’.
• The forth argument is the output report file (the report file is in jslint4java report format).

Copy-paste detection

For copy-paste detection we will use the PMD tool at http://pmd.sourceforge.net/. In order to use it we need 2 things:

• JRE – Java runtime is used for running the the tool. Download it from here and install it. You don’t need the JDK only the JRE.
  Jenkins also include JRE in ‘Jenkinsjrebin’, but using it may provide compatibility problems.
• PMD – Download the archive from the main site and extract in folder ‘C:ToolsPMD’. The required files are ‘commons-io-X.X.jar’, ‘jcommander-X.X.jar’, ‘pmd-core-X.X.X.jar’, ‘pmd-cs-X.X.X.jar’, ‘pmd-javascript-X.X.X’.
• Make sure the java.exe path is in the system environment path.
• Make a file called fix.bat in the “c:ToolsPMD” with the following text:

  @ECHO OFF
  for /F "delims=" %%A in ('dir /b /s %1') do (
  	IF %%~zA equ 0 (
  		echo %%A Fixed
  		echo ^<?xml version="1.0"?^>^<pmd-cpd /^> > %%A
  	) ELSE (
  		echo %%A Ok
  	)
  )

The following script will generate the CPD violations report for .js and .cs files:

REM 'Check for CPD in C# files'
java.exe -Djava.ext.dirs="C:ToolsPMD" net.sourceforge.pmd.cpd.CPD --minimum-tokens %CPDMAXTOKENS% --files * --exclude packages/ --language cs --encoding windows-1252 --format xml > ReportsCSCPDViolations.xml

REM 'Check for CPD in JavaScript files'
java.exe -Djava.ext.dirs="C:ToolsPMD" net.sourceforge.pmd.cpd.CPD --minimum-tokens %CPDMAXTOKENS% --files * --exclude packages/ --language ecmascript --encoding windows-1252 --format xml > ReportsJSCPDViolations.xml

REM 'Check if CPD report files are empty'
call c:ToolsPMDfix.bat Reports*CPDViolations.xml

Other

We can also include the HTML 5 validator that can be run with JRE and do some nice checks of your HTML. However it can not check templates, like Razor.
Another tool is the Css Lint, that check your css files and can be runned with JRE and Rhino.

Visualization

For visualization of the analysis we will use the Violations plugin. The plugin can read the results for JSLint, FxCop, StyleCop and other reports and generate nice graphics for them.
Note: Maven Integration plugin is required with the Violations plugin, but is not automatically installed.
In the Post-Action select ‘Add post-build action -> Report Violations’ and config it like this:

cpd: Reports*CPDViolations.xml
fxcop: Reports*CPDViolations.xml
jslint: ReportsJSHintViolations.xml
stylecop: Reports*.StyleCopViolations.xml

Report

In order to provide this information to the developers we can use Extended Email Plugin for Jenkins. The plugin can be setup to use different email templates, according to your needs. The plugin have default templates. You can download additional ones or create one on your own.
In the template you can include Unit test reports, analysis results, system variables, etc.
In order to use the email-ext select ‘Add post-build action -> Editable Email Notification’. Setup can be:

Project Recipient List: List of email addresses separated by comma (ex: [email protected], [email protected]).
Project Reply-To List: Reply to addresses as the recipients.
Content Type: Html or plain text email type.
Default Content: One of the predefined templates like ‘${SCRIPT, template=”groovy-html.template”}’ or your custom template.
Attach Build Log: You can specify if you want to attach the Jenkins build log to the email.
Advance Options -> Triggers: Specify what triggers the notification.

In order to be able to send email you need to configure in the Jenkins configuration page -> section ‘E-mail Notification’.

Executed Command

The final script for the analysis should be as follow:

Before MSBuild

REM 'Clean Up'
rd /S /Q TestResults
rd /S /Q Reports
md TestResultsHTMLJSReport
md Reports

After MSBuild

REM 'Start FxCop to check for general coding problems.'
"C:ToolsFxCopFxCopCmd.exe" /out:ReportsFxCopViolations.xml /ruleset:=ToolsDotNetCodingStandards.ruleset /file:%WEBPROJECT%/bin/%ASSEMBLIES%.*.dll /gac /ignoregeneratedcode

REM 'Start JSHint to check for JavaScripts coding problems.'
"C:ToolsJSHintphantomjs.exe" C:ToolsJSHintjshint-runner.js %WEBPROJECT%Scriptsapp ToolsJSHintRules.json ReportsJSHintViolations.xml

REM 'Check for CPD in C# files'
java.exe -Djava.ext.dirs="C:ToolsPMD" net.sourceforge.pmd.cpd.CPD --minimum-tokens %CPDMAXTOKENS% --files * --exclude packages/ --language cs --encoding windows-1252 --format xml > ReportsCSCPDViolations.xml

REM 'Check for CPD in JavaScript files'
java.exe -Djava.ext.dirs="C:ToolsPMD" net.sourceforge.pmd.cpd.CPD --minimum-tokens %CPDMAXTOKENS% --files * --exclude packages/  --language ecmascript --encoding windows-1252 --format xml > ReportsJSCPDViolations.xml

REM 'Check if CPD report files are empty'
call c:ToolsPMDfix.bat Reports*CPDViolations.xml

EXIT 0

Note that in the end we do a EXIT 0, because FxCop and some of the others tool can fail the build and this is not our intention.

References

• Code analysis team blog
• PhantomJs