Working with Checkmarx
During the last few years VR Security has been involved with the design and implementation of Checkmarx (a commercial static code analysis tool) for two large enterprises.
The scale of each implementation has been very different with respect to the technical design and adopted workflow. For example, with one of them it was decided to deploy in AWS from the get go and with the adoption by teams/users driven by management; the other was an on-premise solution with a large number of design assumptions made to cater for unknown scan request profiles due the the size of the development community and vast range of chosen technology.
A few initial design decisions (assumptions) were made in both cases:
The default set of scan rules (aka Preset using Checkmarx's terminology) was used
All scan engines were initially configured the same, as the codebase (Line of Codes and languages/framework) for the various teams were an unknown
Each scan engine server only processed a single scan job
The scan request throughput for each implementation was very different as no experience (data points) existed which could be leveraged for an accurate analysis and to forecast. The AWS instance was what would be called relatively normal i.e. less than 100 scans per day, whilst the other exceeded 1000 scans per day.
Adoption of any new solution, especially a security testing tool is always challenging as the benefits are not always obvious for teams with a fast development cadence and the traditional end of cycle rubber stamping experience still being enforced.
There are two obvious adoption methods:
Target certain teams to try out and provide feedback and gradually open up the service to the rest of the organisation; whilst the scale of the solution and associated workflows becomes more suitable for the organisation
The alternative, which could be perceived as uncontrolled, is to allow any team to come forward to onboard and start using
With the first approach the general behave has been that those targeted teams did not want to participate and therefore the adoption and usage of the solution was very light, if any. As for the second approach, the teams coming forward clearly had an interest with improving their understanding of the security profile of their development. The latter reaction is probably music to the ears of the security team, however if this turns out to be an irregular growth pattern, then this causes an operational challenge with scaling out the instance of the tool.
Here are a few details of these types of challenges:
A self service model could be considered as a way forward regarding the RBAC model (aka Checkmarx Organisation Hierarchy) to accommodate the typical cumbersome access request process in large organisations e.g. an extra layer of Companies could be created to allow the team leads to become Company Managers and therefore have the rights to add and remove users from their area.
To expand the capacity of the scan engine server farm, it quickly became obvious that the deployment of the scan engine servers had to be automated. From a value stream mapping perspective you have generally two events to optimise i.e. how quickly you can complete each task and how quickly can a task be picked up and processed after a handover. To address the first point, selecting the native Windows scripting language (Powershell) to create auto deployment scripts for the scan engine servers was an easy win. This journey was shared and recorded.
Optimising the scan results is a must do task and to do this you need to utilise Cx Auditor. The version of this end client piece of software needs to be in sync with CxSAST (server backend). This can become a tremendous chore, as typically all software deployed on end user based systems in enterprises will likely need to be software packaged up by an internal team. Note in most cases deploying an update to the backend will be quicker than going through the packaging process.
Working with vendors within a large enterprise brings its benefits - an example of this was the unexpected invitation to form part of the inaugural EMEA Customer Advisory Forum. Here we spent a whole day with other chosen customers and most (if not all) of Checkmarx's various Product Owners to discuss and provide feedback to shape Checkmarx's long term roadmap. All vendors should conduct such an exercise to help both themselves as well as their customers strive towards a perfect solution.