The Workplace of the Future: Skills, Recruiting and Assessment

Guest Post by Smarterer Co-Founder and CEO Dave Balter

In today’s economy, skills are your competitive edge within the workforce and are critical to career success. Whether you’re a master at Excel, can program in Swift or manage social via Hootsuite determines your career path. But how do you let employers know that you’re good at these specific skills and tools? Or, if you’re an employer, how do you measure which members of your team have the exact skills you need?

Years ago, I was interviewing people for an analyst position. The tenth interviewee looked at me and asked, “Do you know what R is?” I didn’t know what R was (it’s a programming language used by data analysts), and the interviewee responded, “How can you interview me if you don’t know what that is—and how would you know if I’m any good at it?” He was right (although his snarkiness didn’t get him hired), so I called a few assessment companies. The response was astounding: an R test wasn’t available because it was “too new.” It would take months to create and would cost $30-$40k to do so. Plus, since R was open source, the language was continually being improved upon and further developed. At the time, assessments were designed as a set of fixed, never-changing, validated questions. So how do you validate what people are good at in an ever-changing, increasingly fast-developing technology world?

Thus, Smarterer was born. We began with a simple premise of using the crowd to develop our content. Instead of identifying a handful of Subject Matter Experts (SMEs) and hiring them to write all of the questions—which is how traditional assessment companies design tests—we allow anyone in the world to write and even edit questions. That means experts come out of the woodwork, and the test items can naturally evolve as the skill evolves. Does it work? Just two weeks after Apple announced the Swift for iOS, the crowd had created a Swift test on Smarterer.

But it’s what we do with those test items that really makes our platform exciting. We deliver our questions adaptively (similar to the SAT, how you answer determines the difficulty level of the next question), then apply machine learning and Bayesian algorithms to the test taking experience—while each test taker is getting scored, we’re using their answers to inform difficulty levels on each question. There’s a whole bunch of other algorithmic magic—like modeling uncertainty into the test item curves—and the net result is astounding: we can validate anyone’s skill in anything in as few as 10 questions, in 120 seconds or less.

In the past four years, more than 1000 skills and 100,000 test items have been developed in the Smarterer system, from Excel, to English for business, to negotiation—anything you’d consider a professional skill. This crowdsourced database is open to the public to use for free, and partners like Elance, Care.com and University of Phoenix (Apollo) use our API to present assessments in learning paths and professional profile development.

About a year ago, a hiring manager from Expedia told me, “Your crowdsourced tests are great, but they don’t measure exactly what my team does. We do things differently here.” The truth is every organization applies tools and utilizes skills differently, and we began to see the pattern of organizations wanting to adapt tests to match their proprietary processes. We learned that there was a massive need for organizations to understand who is good at what within their company, and what skills might be missing. So we began developing a platform that would allow an organization’s own employees to develop test items, to capture their proprietary knowledge that could measure their own organization’s skill inventory and skill gaps. In January of 2014, Flock was born (we love owls, but Parliament didn’t really roll off the tongue…).

Flock has now rolled out to nearly a dozen Fortune 500 organizations and is often implemented alongside internal training and learning systems, providing employees with clear learning paths. For example, one large financial institution that we work with has a learning management system with more than 400 videos on the Agile software development methodology. A true paradox of choice, employees are often confused about which video will help them grow in their career. With the implementation of Flock, they can take a quick assessment (developed by other employees), and it will tell them which five videos to watch.

In another instance, an organization utilized Flock to capture the knowledge of 6,000 customer service representatives and their specific day-to-day actions. The complete assessment was then delivered to trainees after a 14-week course to understand the impact of the training and where individuals fit within the overall customer service team. In another case, an organization utilized the tool to help roll out their new “beliefs” to more than 300,000 employees—where situational questions can pinpoint someone’s understanding of the beliefs, and data analytics can be applied to understand the differences between perceptions at each employee level or across regions.

The market is quickly moving to where the majority of learning is going to be done online, and quantifying a wide variety of skills has become critical to an employee’s professional experience. It used to be possible to get away with “I took the course.” Now, it’s “which course should I take, did it have enough impact for me to be able to say it was worth it, and can I prove that I’m now an expert in this area?”

For organizations, this has never been more important due to the speed of change, and the role skills play in success. Simply put, staying ahead requires your team to change as fast as your business does.