One of my responsibilities at Acquia is overseeing the development of our certification program. We’re creating this program to establish baseline credentials for working with our commercially supported distribution of Drupal (codenamed Carbon) and our Spokes and Caliper network services. Since Carbon is a distribution of Drupal core and a selection of modules, Acquia certification will judge facility to work with Drupal independent of our distribution.
I was not surprised that the Drupal community is suggesting caution when it comes to our effort and the value of certification in general. I’ve been a certification skeptic since I first started working with Certified™ technical staff. There are too many “paper” certifications in the IT community, and I’m not interested in adding another to the pile. To tell the truth, I’m a bit of a skeptic when it comes to technical training as well—I remember someone close to me telling me so few people failed his classes because “their boss didn’t send them to fail.”
I’ve had to come to grips with my skepticism over the past few years as I’ve had managers ask me to get Certified™ and I’ve developed and delivered training programs. What I’ve realized is that it’s not really about the coursework, the instructor, or the exam. It’s about the student. There are a lot of different approaches certification, and different attitudes to take with you to a training class. There’s also a lot of ways to think about the value of being Certified™.
In any given field, the available talent tends to follow a bell curve. Way over at the right of the curve, certification probably doesn’t add much value. People at this level will either be well enough known, or have a strong enough network, that they won’t need much in the way of outside credentials to establish themselves. Way over at the left, certification probably doesn’t add much value for the candidate (I’m assuming they’d fail), but it does serve to weed them out.
In the middle of the curve is where it becomes more interesting. Here, certifications help distinguish one from the other on an objective scale. The scale is probably not the most important one for selecting an employee or vendor, but it does help to quickly narrow the focus (and thus the effort) for vetting soft skills and other intangibles. The key is designing the scale to minimize the false positives and false negatives.
Robert Douglass, who owns the certification effort on my team, shared his success with Java certification in the Acquia team blog. He used the Sun Certified Java Developer program to organize his efforts to learn Java. Essentially, he moved himself along the bell curve with perspective employers by getting certified. The certification helped two ways—it provided him with a path for his learning and established to the Java community that he had the basis for becoming a successful developer.
How do we design the scale for our certification so that it doesn’t become a “paper certification” (too many false positives)? The key challenge is in identifying what we’re testing for an how to evaluate those skills. As an example, we could test “book knowledge” with questions like:
What is the purpose of
hook_enable?
a. install the current version of the database schema
b. perform module setup tasks
c. called before a node is shown on a form
d. defines a set of access restrictions
and with sufficient cramming a certification candidate could pass a test like this with no understanding of what a hook is, or why they’d care. This is a great way to devalue the differentiator by flooding the market with “paper” certifications. This is not the certification we want when we build Yellow Jersey.
I want a certification that tests more than just facts and memorization; an exam that tests the knowledge and wisdom that only comes from practical experience. I have a few ideas about this, but I’m not sure of the best way to go about it. We’ve met with some experts, and I’ve heard a lot of advice about emulating Cisco or Sun. I haven’t studied ether, but I understand that they involve not only multiple choice tests but also a practical component. I think this is necessary, but I’m worried that the cost may be prohibitive in the short term.
I think there’s also a lot we can do in terms of developing the multiple choice questions to probe a deeper level of understanding. Questions on performance, for example, that require the test taker to evaluate a set of requirements against a set of deployment and caching options. Or questions that ask for an evaluation of a segment of code to explain what it’s doing and why.
Keep an eye on the Yellow Jersey to follow how it works out.
