This is the second part of my post about static analysis setup on iOS for test results and code coverage, PMD analysis, duplicate code trends and source lines of code count metrics. For this part, I'll be referencing this repository. If you're interested in what are those metrics or what are the motivations behind this, be sure to check the first part of this post.
We'll be using xctool as our build system for this. Advantages of this over the regular xcodebuild are well described by Mattt Thompson on NSHipster. Basically, we're gonna need compilation output in reports (namely junit and json-compilation-database formats) and xctool already includes these reporters.
So, to setup xctool with your project, you will need to install the script on your computer and add a
.xctool-args file on your repository. Refer xctool documentation for information about this, but you'll need something like this.
Test results and Code coverage
You will need something like this on your CI server job in order to get test results:
xctool -reporter junit:Build/junit.xml clean test
This will run your test target and store the results on
Build/junit.xml in the classic junit format. Most Jenkins CI installations already come with post-build actions for JUnit test result publishing, just setup one of those and you'll get something like this:
If you happen to run with this error on your CI server:
ERROR: Cannot find schemes. Please consider creating shared schemes in Xcode.
It means that you'll need to set your scheme to shared on Xcode. You can do this on Product > Schemes > Manage schemes.
Getting the code coverage is a little harder. First, you'll need to turn on two flags on your project debug main target:
Generate Test Coverage Files and
Instrument Program Flow. Then, you'll need to run something like this:
xctool clean test OBJROOT=./Build
This will store the test coverage files on a path I will describe on the next script, but first you need to install GCOVR (I always install those scripts on
/opt/scripts/), a tool that will translate these results into Cobertura XML format. Then, you can get the cobertura results with this:
/opt/scripts/gcovr -r . --object-directory Build/TestingApp.build/Debug-iphonesimulator/WeatherApp.build/Objects-normal/i386 --exclude '.*Tests.*' --exclude '.*main.*' --xml > Build/coverage.xml
See that we're looking inside
Debug-iphonesimulator, the target where we enable the two flags. On Jenkins CI, you will need to install Cobertura plugin, setup a post-build action and publish those results. You'll get something like this:
If you're getting blank results, at the time of this writing, there's a bug with Xcode 5. In order to get pass through this, you will need to call
__gcov_flush() from your application using a test hook, details on how to do this are shared on this StackOverflow question.
PMD analysis results
First, install OCLint. This tool will need a
json-compilation-database which is just the list of commands that xcodebuild runs. You can get those with xctool using the following:
xctool -reporter json-compilation-database:compile_commands.json clean build
Then, run OCLint:
oclint-json-compilation-database -e Pods/** -- -max-priority-1 20 -max-priority-2 50 -max-priority-3 100 -report-type pmd -o Build/oclint.xml
There we're excluding Pods directory, because we're not interested on those sources. Max priority values are set in order to make the build fail if those are exceeded.
In order to publish those results on your Jenkins CI, you'll need to install PMD plugin, setup a post-build action and publish those results. Then you'll get something like this:
Duplicate code trends
This one is fairly straightforward. You'll need to download PMD and Objective-C-CPD-Language (PMD version 4.2.5 and Objective-C-CPD-Language version 0.0.7 worked for me). Then run something like this:
java -Xmx512m -classpath /Applications/pmd/lib/pmd-4.2.5.jar:/Applications/pmd/lang/ObjCLanguage-0.0.7-SNAPSHOT.jar net.sourceforge.pmd.cpd.CPD --minimum-tokens 10 --language ObjectiveC --encoding UTF-8 --format net.sourceforge.pmd.cpd.XMLRenderer --files WeatherApp --files WeatherAppTests > Build/cpd-output.xml
In order to publish those results on your Jenkins CI, you'll need to install DRY plugin, setup a post-build action and publish those results. Then you'll get something like this:
Source lines of code count
First, install CLOC. Then run something like this:
cloc --by-file --xml -out=Build/cloc.xml WeatherApp
This will store the source lines of code count on a xml file but in order to get those published by Jenkins CI, we'll need to translate that xml to SLOCCount format with this:
xsltproc Utils/Sloccount-format.xls Build/cloc.xml > Build/cloccount.sc
xsltproc already comes preinstalled on OSX and you can grab an example of
Sloccount-format.xls here. The reason why I'm doing this is because I couldn't get
SLOCCount work on OSX Mavericks for some reason, so if you can solve this, please let me know.
In order to publish those results on your Jenkins CI, you'll need to install SLOCCount plugin, setup a post-build action and publish those results. Then you'll get something like this:
I like to have my scripts inside a directory on the project repository, but that's up to you. We surely can do some optimizations here, like for example run
xctool test only once and attach all the needed
-reporter parameters. I let you experiment with that.
Hopefully you get everything right. But if not, just send me an an email to firstname.lastname@example.org. Good luck and thanks for reading.