Static Analysis on iOS - Part II

This is the second part of my post about static analysis setup on iOS for test results and code coverage, PMD analysis, duplicate code trends and source lines of code count metrics. For this part, I'll be referencing this repository. If you're interested in what are those metrics or what are the motivations behind this, be sure to check the first part of this post.

The following assumes that you have basic knowledge about Jenkins CI and Linux Shell scripting.


Build system

We'll be using xctool as our build system for this. Advantages of this over the regular xcodebuild are well described by Mattt Thompson on NSHipster. Basically, we're gonna need compilation output in reports (namely junit and json-compilation-database formats) and xctool already includes these reporters.

So, to setup xctool with your project, you will need to install the script on your computer and add a .xctool-args file on your repository. Refer xctool documentation for information about this, but you'll need something like this.

Test results and Code coverage

You will need something like this on your CI server job in order to get test results:

xctool -reporter junit:Build/junit.xml clean test

This will run your test target and store the results on Build/junit.xml in the classic junit format. Most Jenkins CI installations already come with post-build actions for JUnit test result publishing, just setup one of those and you'll get something like this:

test_results

If you happen to run with this error on your CI server:

ERROR: Cannot find schemes. Please consider creating shared schemes in Xcode.

It means that you'll need to set your scheme to shared on Xcode. You can do this on Product > Schemes > Manage schemes.

Getting the code coverage is a little harder. First, you'll need to turn on two flags on your project debug main target: Generate Test Coverage Files and Instrument Program Flow. Then, you'll need to run something like this:

xctool clean test OBJROOT=./Build

This will store the test coverage files on a path I will describe on the next script, but first you need to install GCOVR (I always install those scripts on /opt/scripts/), a tool that will translate these results into Cobertura XML format. Then, you can get the cobertura results with this:

/opt/scripts/gcovr -r . --object-directory Build/TestingApp.build/Debug-iphonesimulator/WeatherApp.build/Objects-normal/i386 --exclude '.*Tests.*' --exclude '.*main.*' --xml > Build/coverage.xml

See that we're looking inside Debug-iphonesimulator, the target where we enable the two flags. On Jenkins CI, you will need to install Cobertura plugin, setup a post-build action and publish those results. You'll get something like this:

code_coverage

If you're getting blank results, at the time of this writing, there's a bug with Xcode 5. In order to get pass through this, you will need to call __gcov_flush() from your application using a test hook, details on how to do this are shared on this StackOverflow question.

PMD analysis results

First, install OCLint. This tool will need a json-compilation-database which is just the list of commands that xcodebuild runs. You can get those with xctool using the following:

xctool -reporter json-compilation-database:compile_commands.json clean build

Then, run OCLint:

oclint-json-compilation-database -e Pods/** -- -max-priority-1 20 -max-priority-2 50 -max-priority-3 100 -report-type pmd -o Build/oclint.xml

There we're excluding Pods directory, because we're not interested on those sources. Max priority values are set in order to make the build fail if those are exceeded.

In order to publish those results on your Jenkins CI, you'll need to install PMD plugin, setup a post-build action and publish those results. Then you'll get something like this:

static_analysis

Duplicate code trends

This one is fairly straightforward. You'll need to download PMD and Objective-C-CPD-Language (PMD version 4.2.5 and Objective-C-CPD-Language version 0.0.7 worked for me). Then run something like this:

java -Xmx512m -classpath /Applications/pmd/lib/pmd-4.2.5.jar:/Applications/pmd/lang/ObjCLanguage-0.0.7-SNAPSHOT.jar net.sourceforge.pmd.cpd.CPD --minimum-tokens 10 --language ObjectiveC --encoding UTF-8 --format net.sourceforge.pmd.cpd.XMLRenderer --files WeatherApp --files WeatherAppTests > Build/cpd-output.xml

In order to publish those results on your Jenkins CI, you'll need to install DRY plugin, setup a post-build action and publish those results. Then you'll get something like this:

duplicate_code

Source lines of code count

First, install CLOC. Then run something like this:

cloc --by-file --xml -out=Build/cloc.xml WeatherApp

This will store the source lines of code count on a xml file but in order to get those published by Jenkins CI, we'll need to translate that xml to SLOCCount format with this:

xsltproc Utils/Sloccount-format.xls Build/cloc.xml > Build/cloccount.sc

xsltproc already comes preinstalled on OSX and you can grab an example of Sloccount-format.xls here. The reason why I'm doing this is because I couldn't get SLOCCount work on OSX Mavericks for some reason, so if you can solve this, please let me know.

In order to publish those results on your Jenkins CI, you'll need to install SLOCCount plugin, setup a post-build action and publish those results. Then you'll get something like this:

sloc_count

Closing

I like to have my scripts inside a directory on the project repository, but that's up to you. We surely can do some optimizations here, like for example run xctool test only once and attach all the needed -reporter parameters. I let you experiment with that.

Hopefully you get everything right. But if not, just send me an an email to renzo.crisostomo@me.com. Good luck and thanks for reading.

comments powered by Disqus