Feedback and the Future of the Freelance Marketplace

The last time I stayed in an Airbnb, the host made me pick up the apartment keys from a cashier at a convenience store several blocks away. Naturally, when it came time to rate my stay on the site, I docked a few points for the check-in process since the host should know it was pretty inconvenient. (Although I did score some Tastykakes waiting in line.) AirBnB’s ratings system is a mandatory component to build trust with their customers. So why don’t we see similar ratings in creative industries? Wouldn’t more feedback help ensure quality in a marketplace economy?

When selecting someone for creative work, it’s rare to find data that helps an employer or buyer know what to expect. Sure, Etsy vendors have star ratings, but those reviews typically capture how someone delivers on a specific product that the consumer sees online beforehand. The world of a creative freelancer is different. Unlike that basket you purchased on Etsy, every project for a freelancer is new. They’re forced to market their skills more than their projects, which means feedback becomes more personal than usual. That gets even trickier when freelancers receive feedback from an editor or client who may never even meet them face to face.

As a guest on AirBnB, I get rated after every stay. I currently hold a 5-star rating (no biggie), but I probably wouldn’t freak out if I got dinged for checking out late or spilling wine on a bathrobe. For professional creatives, though, a few bad ratings impacts their career.

As a member of the team that manages Contently’s global talent network, part of my job is to think about how the creative process affects contributors. Imagine reporting for the Wall Street Journal for 30 years, working with a brand for the first time, and finding out you were removed from the account because a marketing manager wasn’t impressed with your work. You’re probably not going to want to work with brands anymore.

So this puts freelance industries in a pickle. On one hand, any successful marketplace needs data on people and products to ensure quality. But that data has to be gathered and conveyed thoughtfully. Most people can identify a clean apartment, but evaluating an article or infographic is much more complex and subjective.

At Contently, we tried to find a way to make the process more objective. Last year, we started asking editors to provide quantitative feedback on each project once it was submitted. Our rubric includes five categories: grammar, structure, sourcing, responsiveness, and etiquette—each with a five-point scale. These ratings influenced our staffing decisions for clients, but we were still weighing whether to notify contributors of their performance on every story. Ideally, such a move would incentivize freelancers to submit better work.

We’ve decided to move forward with the plan because of one reason: accountability. Contently was built on the trust between clients and freelancers. If we all of a sudden doubt their ability to maintain that relationship, even in the face of candid feedback on their performance, it goes against what has worked so well for us to date. If a contributor receives a 3 out of 5 on responsiveness or etiquette, chances are she will discuss the score with the editor and pay more attention to those areas for the next story.

After more than a year of collecting data, the results are promising. Once contributors are notified of their ratings, scores on subsequent submissions improved.

Criteria averages before surfacing scores to contributors:

Criteria averages after surfacing scores to contributors:

Accountability works both ways too. Before we opened up contributor ratings, most of the feedback we received from clients and editors was pretty vague, along the lines of, “Bill isn’t cutting it for us, his last few stories need a bit more editing than others.” We’d press for more information, but by that time, minds were already made up. Instead of working through the disconnect when they start to surface, the client would remove Bill from the team of contributors.

This new system generates instant feedback and a chance for contributors to understand where they can improve. Criticism may disappoint or surprise them at first, but it’s better than finding out you’ve been removed from a team based on vague, anecdotal feedback. In the past, if an editor or marketer didn’t agree with, say, a structure choice, there was no debate or opportunity to compromise. Now, we expect the change to open the door for contributors to explain their decisions and have a productive dialogue.

New freelance marketplaces are emerging for just about every creative field you can think of. And a lot of these places are helping connect freelancers with work they normally wouldn’t be able to find. But to sustain these marketplaces, including Contently, we have to make sure that we’re setting up clients and creatives with an infrastructure that leads to better quality work. Employers review the performance of full-time employees all the time. For freelancers to succeed, they shouldn’t be treated any differently.