Hunting the Best: Thoughts On Judging, Awards, and their Agency

Before we get into this I want to start by addressing an elephant in the room, but something I believe to be a fundamental truth, even if it’s not trendy right now: judging is an essential human act. It is wholly natural, from your surroundings to the things you consume, judging is instinctual – it’s an act that kept our ancestors alive long enough to evolve, and climb to the top of the food chain. It’s how we decide what we like, or don’t. I know we all like to pretend that judging each other is wrong, and we should all be Kumbaya about everything, and while judging people for traits they can’t control like their skin color or body type is in fact lame and something we should evolve past, let’s not forget that judging is something we all do countless times every day. This is true about competitions as well, so it’s no surprise that humans have found ways to compete for awards and crown winners since the beginning of time. And it should be no surprise that there are those of us doing that for weed, as well.

Now that that’s settled, and not to sound like too much of a weed snob (though I am), by this point I’ve judged just about every major cannabis competition in the States. From the Cannabis Cup to Zalympix, it seems there’s very little consensus around how this should be done event to event – which is likely why there are so damn many of them now, and why each does it slightly different. New ones pop up to fill the gaps they saw in the old ones, and so on. While I do think some people are doing great work, and High Times is of course the originator of this model in our space, I feel inclined to discuss the differences between them to give those of you at home a better understanding of how this process works – and hopefully call some of the other organizers up to a higher level.

Right now the three most visible competitions are – in my mind – High Times’ own Cannabis Cup, the Emerald Cup, and Zalympix. While all are primarily based in California, both the High Times competition and Zalympix tour the country and sometimes world, doing smaller regionally-focused competitions until a national or international one is possible sometime in the future. These competitions all aim to be compliant with local regulations, and restrictions can be heavy. Then there are other notables as well, like Ego Clash, which holds a competition in Barcelona as well as Northern California, and Chalice, which has now evolved into the ‘Best of’ competition series, which has already held a few events on the East Coast. Famed pot-critic Jimi Devine even has his own, dubbed the Transbay Challenge, after its initial pilot pitting SF and Oakland’s best producers against each other, and more are popping up every day like mushrooms. Attending and judging these have become a solid chunk of my calendar each year, and while they’re all fun because I get to smoke a ton of weed, they all take a solid deal of work to complete. I’m not complaining here – it’s an honor every time someone asks – but hopefully addressing this all publicly will help improve the entire process for everyone, and potentially put organizers onto some issues that they could solve with their own.

You see, these events don’t just have different trophies, they each have very different formats for judges. While I’m not going to point out the nuanced differences between them individually – or call out any I think are doing it wrong, per se – I do want to call attention to some distinctive details that I believe can significantly dictate the outcome of the event. These include:

• Blind Judging
• A Gifting / Buying Mix
• Ample Time with the Entries
• The Categories Measured
• Using the Data vs. Feelings

Before we get to all that though, let’s talk about the merits of a competition in the first place.

Trophies That Matter

The first thing that anyone should understand about any particular competition is that its pinnacle of success will only ever be as strong as the trust in the hosting agency. The organizer is what gives it its prestige – it’s the reason why people are swayed by it. If no one knows, or trusts, the competition’s organizers then the trophy is just a hunk of metal. Sure it feels good for the winners at the moment – everyone likes to win something – but the reason why most brands enter these competitions is because they want the award to have some sort of impact on their brand. Maybe it’s increased sales, maybe it’s increased awareness, but none of these brands are just throwing away their products for no reason. If no one has heard of the issuing agent, how much merit does that trophy actually hold? 

Next, the breadth of the competition is a cornerstone. If the contestants only represent one sect of the community it will only be respected by that sect, so having an all-encompassing competitor list is paramount. I appreciate that we all think we’re the best or coolest or most important, but the truth of the matter is most of us represent very niche areas of the market, and very few are fully representative. I should clarify that this is extremely hard to do, and impossible legally because of so many potential contestants limited licensure. We try to do this ourselves with our media coverage but there is just SO much happening in this space that even there it’s almost impossible to do – but that doesn’t mean you shouldn’t try. If a competition doesn’t provide a true read of the entire market then it’s not really a championship, it’s a divisional at best.

Finally, who is actually judging? I know everyone’s going to expect me to say ‘it should be me and my friends, we smoke the most!’, but in truth I believe that an effective judging class should be representative of the entire market, just like the breadth of the competition. If you only have Candy Gas guys judging then you know who the winner will be before the competition starts. If I’m the only judge it will likely be something Z related – or an orange varietal, which I get chewed out for even liking all the time. The point is that by having a variety of judges that each represent a different walk of life you have a much better chance of providing results that the whole community can agree with. This is really what you should be aiming for. The more people who say what you chose to win is lousy, the less they’ll respect what you’re doing, and nobody wants that.

Conditions of Judgement

Now for the fun stuff – where things really start to go off the rails. The main differences in everyone’s competitions come down to the actual judging process. While there are many different opinions about the process judges should go through when examining and scoring their samples, these are where the details I mentioned earlier come in. I don’t think anyone has effectively nailed all of these conditions yet – some simply due to compliance issues, like with our current People’s Choice model – but in a perfect world, I believe a perfect competition would satisfy all of the below requirements:

Blind Judging

Blind judging essentially means that the judges are to score the products based on the strength of the product alone, without the sway of branding or marketing. It means you just see every bud through the same lens – that is, a bud in a clear jar with no bells or whistles to woo you. When the Cannabis Cups began, this was how it was done. It was a true battle of the buds – it didn’t matter who grew it, or where it came from, or if the genetics were trendy at the moment – it was about which bud was the best in a sea of buds (the goal of which is not that dissimilar from what we have in the market today). While legal compliance in most states, which says that consumers need to be able to see and understand the product before they consume it, prevents this from happening in most legal markets, this is probably the most important aspect in having a truly unbiased competition. 

A Gifting / Buying Mix

This one is going to be polarizing, but I believe all competitions should have a mix of both ‘earned’ judges and regular consumers who purchase the kit. I believe they should include industry newbies as much as the vets. And I believe they should include consumers – especially those who have little understanding of the grow or extraction process themselves – because if we want the awards to be representative of everyone, everyone’s voice should be counted. Granted we don’t have the budget to do this like a government election and allow every single person a voice in this process – it would simply take forever and there’s not enough of each product to go around and have the producer stay in business  – however, if we’re trying to judge something on behalf of everybody, doesn’t it make sense that everybody’s perspective should be included? Now I know people are going to say ‘leave it to the experts’, but I ask you, are the experts representative of the majority of cannabis consumers? We all know that not to be true. Most of the OGs only want to smoke OG, while spray packs are doing the craziest numbers in the market right now – do we just want to satisfy the heads, or do we really want to understand what the kind of the market is right now?

Ample Time with the Entries

This is, in my opinion, another one of the most important factors. While I appreciate the marathon judging that happens at some of these events – that is, where you have to go through all the samples during the course of a day, or the event – we MUST remember that the most important thing to judge in this should still be the actual effect of the product, not just the flavor. We only really started smoking for flavor a little over a decade ago, but people have been consuming for the effect forever: we can’t get away from what matters with these things. In a perfect world I think judges should have at least a day with each sample – this is more about making sure there’s no cross contamination in the high than anything else. Obviously we can’t expect heavy smokers to just smoke one thing a day, but if each judge starts their day with a sample and saves the next for the next day that’s probably the closest they’ll get to sober, and thus the closest to a completely fair judgement. When we marathon the colors bleed way too much – after a few samples you’re dealing with entourage highs, not just the effect. This is of course not possible with giant competitions though, so I do believe that regionals into a championship is the most effective way to do this in today’s landscape.

The Categories to Measure

This has probably the widest variance event to event. Some want scores for several categories, as well as comments, and others just want to know what your top few favorites are. Most use a spreadsheet system and have you score against things like initial look, smell, flavor, burn, and effect. The thing is, I’m not sure all of these should be weighted the same, as the effect and the burn are in my opinion the two most important to the actual consumption experience, though the look and smell are what typically make the sale in the marketplace. Maybe effect and burn should be counted twice while look and smell are only counted once? But then we’re weighing the competition against the details that actually make a sale today. This is definitely an opinion based matter, but if you’re not measuring all of these individually I believe it will be hard to effectively tally the results – especially if you have real variance amongst the judges, or want to award any smaller trophies like the best tasting or best smoking, etc.

Using the Data vs. Feelings

Finally, how are the scores actually tallied? Now I’m admittedly a data nerd, but this is another one that I don’t think there should be any questions about: if all the judges submit complete unbiased reports on all the entries that should be the only thing that determines the winners. Some competitions hold a final process where judges compare their own findings against those from the other judges in an attempt to create agreement amongst the judges, and for them to highlight any favorites they had to those that may have missed it during their process. To me that’s a flawed premise – you should stick to the numbers and keep conjecture out of it, though I am always happy to sit around a table and yell at each other. However, sticking to the data guarantees the integrity of your competition. I don’t necessarily think it should be publicly released, but it does need to be saved in case anyone wants to call the events results into question. People LOVE to say that the competition is fixed when their favorites lose – this won’t entirely stop them from saying that, but it will eliminate any doubt in any rational person’s mind when you can pull up the data at any time and address each judges findings individually. Again though, don’t walk into the firing squad of giving all that info out to everyone.

The Final Score

Now I’m not here to tell you which guy sucks, or crown a competition king – although, I do work for High Times, so obviously that answer is the Cannabis Cup if you’re asking, but I digress. 

The long and short of it is that while most of these events are a good time for attendees, and hold some degree of clout for the winners, the truth of it is that they’re always a business themselves. They’re either revenue generators, or near cost-free marketing for a larger entity, so it’s true they’ve all got some other motive besides just wanting to know who the best is, and most have some degree of partiality. 

This is why media has historically been the one to hold the events that generate the most attention: we have to be Switzerland in the marketplace, and while of course we as humans all have preferences, if Switzerland is the governing body of a competition, hopefully it’s something that everyone will have faith in. We are used to dealing with sponsors, selling them advertising, and maintaining our objectivity – which is not as easy for event producers who survive off of those fees specifically. This has proved effective across nearly every category, from the Oscars to the weekly Billboard lists. That’s not to say it’s not a revenue generator for us too – and in a world with rapidly shrinking margins, events and features like this have been a lifesaver for many of us – but all areas of this space are getting crowded, and all money wells are drying up. 

While I’m not trying to say the media should be the only ones to hold these types of awards (that would be incredibly self serving of me), I am inclined to ask what merit these new pop-ups have, and why their trophies should be valued. If you’re just some guy trying to make money the same way a bunch of other people who are established already are, what makes you special?

Hopefully now you new jacks will be able to respond with how you’ve satisfied the above requirements, and some other reasons why your process is better, or more authoritative. If so, I’ll look forward to checking yours out too, but forgive me if I don’t just take your word that you’ve got ‘the new best thing’. I’ve heard that dozens of times before.