How to Set Up Your Judging Process So Judges Actually Enjoy It | Dapple Blog

Introduction

Most awards programmes spend months thinking about how to attract entries. Very few think about what happens when those entries arrive on a judge's screen.

The result is a judging experience that's often chaotic, inconsistent, and a drain on the people you've asked to give up their time. Judges get emails with spreadsheet attachments. They receive PDFs with no context. They're handed a hundred submissions with no scoring guidance and a deadline that's a week away. And then they're asked to come back next year.

Good judges are hard to find and easy to lose. The way you set up your judging process is the difference between a panel that feels valued and one that quietly declines to return.

Here's what actually makes a difference.

What Is a Good Judging Process for a Creative Programme?

A good judging process gives each judge or reviewer a clearly scoped workload, consistent criteria to apply, and a platform that doesn't require them to chase you for information. It removes ambiguity up front and gives you, the programme coordinator or awards manager, a clear view of progress without having to chase anyone.

That's the short version. The longer version involves five things most programmes get wrong.

Why Judges Drop Out (and How to Prevent It)

Judge drop-out is almost always a process problem, not a commitment problem. People agree to judge because they care about the field. They drop out because nobody told them how many entries they'd be reviewing, what they were being asked to score, or where to actually do the scoring.

Ambiguity is the main culprit. When judges don't know what's expected of them in terms of workload, criteria, timeline, or format, they procrastinate or delay.

The fix is front-loading clarity. Before the judging window opens, every judge should know:

• How many entries they're responsible for
• What categories or types of work they've been assigned
• What scoring criteria apply and what each score means
• Where and how to submit their scores
• Who to contact if something's unclear

This isn't complicated but requires a little bit of foresight on the admin side to communicate all this before the window opens, not during.

How to Divide Entries Among Judges Fairly

Assigning entries to judges is one of the most underrated parts of programme management. Done badly, it creates uneven workloads, conflicts of interest, and scores that can't meaningfully be compared.

The starting point is deciding what kind of assignment model you're using. The three most common are:

• Single reviewer per entry — faster, cheaper, but introduces more variability. Works for high-volume programmes where speed matters more than precision.
• Panel consensus — every judge reviews every entry. Produces the most defensible outcomes but is only practical for smaller shortlists.
• Split panel with cross-review — entries are divided between judges, with a subset seen by multiple reviewers to calibrate scoring consistency. This is the most balanced model for mid-sized programmes.

Whatever model you choose, document it. If a finalist or unsuccessful applicant ever asks for feedback or why they were scored a certain way, you need a coherent answer.

On conflicts of interest: ask judges to flag them proactively, not reactively. A judge who reviews the work of a colleague, former student, or current collaborator could be a liability. Again, build this flag into your process before the window opens.

Why Judging Categories Matter More Than You Think

Categories aren't just organisational tidiness. They're what allow different judges to apply meaningful expertise to the work in front of them and allow you to run a programme at scale.

The common mistake is creating categories that work for entrants (photography, illustration, animation) but don't translate into judging assignments. A judge with a background in editorial photography probably shouldn't be scoring motion graphics. A poet shouldn't be the primary reviewer for a sound design submission.

Design your categories with both applicants and judges in mind. The categories that attract entries and the categories that enable expert review are often not the same thing. Reconcile the two before your call goes live.

For large programmes covering multiple regions or countries: separate entries by geography before assigning them to judges. This does two useful things. It allows you to assign judges with relevant regional knowledge or industry context. And if you want to actively prevent hometown bias, it gives you a clean mechanism to route entries away from judges based in the same location.

What a Scoring Rubric Should Actually Do

A rubric exists to make individual judgement transferable. Without one, you have ten judges applying ten different interpretations of “excellence” and a shortlist that may reflect process inconsistency as much as genuine quality.

A good rubric defines:

• The specific criteria being scored (originality, technical execution, impact, relevance to theme etc)
• What each score level looks like in practice, not in theory
• The weight applied to each criterion, if any
• Whether judges are expected to write comments alongside their scores

That last one matters more than organisations tend to assume. If you're offering feedback to entrants, even light-touch feedback, your rubric has to support that. Judges can't give useful feedback without clear criteria to anchor it to. And entrants can't act on feedback that isn't grounded in the stated criteria.

One practical note: keep the rubric to five criteria or fewer. Beyond that, judges start averaging everything out anyway, and the additional granularity is mostly noise.

What to Look for in Awards Management Software

If your judges are currently receiving a spreadsheet, a folder of PDFs and maybe a shared Google Doc, the judging experience you're offering is probably below par.

What judges actually need from a platform is a single place where their assigned entries appear, their scoring interface is ready to use, and their progress is saved automatically. They should not need to download anything, chase you for access, or reverse-engineer a spreadsheet that someone else built. It should be crystal clear and not rocket science to operate.

The features that make the biggest practical difference:

• Centralised assignment — submissions land in judges' queues automatically, based on the assignment logic you've set up. No forwarding emails, no shared drives, no confusion about who has what.
• Built-in scoring — rubric criteria appear alongside each submission. Judges score directly in the platform. No separate form, no back-channel email.
• Anonymisation — if your programme uses blind review, the platform should handle this natively. Not a workaround, not a manual redaction step.
• Progress visibility — you should be able to see, at a glance, how far each judge has got. Not by chasing them — just by looking at a dashboard.
• Remote access — judging panels are increasingly distributed. Your process should work whether judges are in the same room or on different continents.

Dapple is built for exactly this kind of programme. Judges get a clean interface with their assigned entries, scoring built around your rubric, and nothing they don't need. Programme coordinators get real-time visibility without the constant email thread.

The Bit Most Programmes Skip: Briefing Your Judges

Entry guidelines. Rubrics. Assignment logic. Platform access. All of that can be perfect — and still fall apart if judges haven't been properly briefed before they start.

A judge briefing doesn't need to be long. It needs to cover:

• The programme's purpose and what you're looking for
• The scoring criteria and what each level means
• The timeline: when the window opens, when scores are due, and whether there's a moderation stage
• Who to contact for technical issues and who to contact for scoring questions
• Any specific sensitivities — work that may be distressing, topics that require particular care

Five minutes of reading before the window opens is worth hours of confusion during it. Write the briefing as if the person reading it has never judged anything before.

A Note on Feedback

If your programme offers feedback to unsuccessful applicants, that's a significant commitment. Make sure your judges are aware they are required to do this from the outset.

Feedback requires judges to write in complete sentences. It requires enough detail to be useful without being so detailed it takes twenty minutes per entry. It requires your rubric to be specific enough to generate actionable commentary. And it requires someone to review the feedback before it goes out, because judges occasionally write things that they shouldn't!

If you're offering feedback, say so explicitly in your entry guidelines, build the time into your judging window, and set the expectation with judges before they start. Don't assume it'll happen naturally. It won't.

The Simple Version

Set entry guidelines. Assign entries thoughtfully. Give judges a rubric they can actually use. Put everything in one place. Brief them before the window opens.

None of this is complicated. But it does need to be done deliberately and before the entries arrive, not while you're trying to manage the panel at the same time.

The organisations that get judging right are usually the ones that treat the judge experience with the same care as the applicant experience. Your judges are doing you a favour. Return it.