VF48Hours Forum

Cardinal
From: Christchurch
Since: Apr 2011
Posts: 182

Feed for this Topic

Why is National Finalist selection the murkiest part of the judging process?

So I think the vast majority of regulars were comfortable with the competition getting rid of automatic regional representation last year, and likewise we all know that the judging in this competition will always be somewhat inscrutable. But I must say that the available information around how the national finalists are picked has me a bit tempted to reach for my tinfoil hat.

The city judges are named in each city - no problem, they're always a clued up bunch. The people that judge the national finalists are always named, and they're a who's who of international film experts, no worries. But in between, apart from knowing that PJ picks 3 or 4 favourites of his to be national finalists, there seems to exist some kind of Shadowy Cabal picking the non-wildcard national finalists.

All that the info on the website says about it is this:

(1) The Best Shorts from Around the country become the Final Pool.
(2) From this final pool the Grand Finalists are Chosen to compete in the Grand Final along with PJs Wildcards (note: City Winners do not automatically make the Grand Final)

If it stands to reason that the city judges pick a few likely candidates from their city to send up the chain to complete step 1, who receives these films and completes step 2?

I'm not suggesting that these people, whoever they are, are not also the highly clued-up experts we have come to see judging city winners and nationals. I also expect that they're totally impartial. But it strikes me as rather odd that we don't get to know who they are. The only reason I can think of to NOT name them (in the same way that we hear who the city and grand winner judges are) is that there's something somehow controversial (or at least that seems controversial) about who performs this stage of judging.

Anyone?

Reply

dmanthei
From: Christchurch
Since: May 2011
Posts: 418

It goes off of who has the best rating in the Screening Room.

2014 - Robocrop - "Pads of Passion" RomCom (Best Original Score - Christchurch)
2013 - Robocrop - "The Price of Pilsner" Horror
2012 - Robocrop - "Promises" Crime - DQed (Nominated Best DQed Film)
2011 - Robocrop - "Protocol" One Room (Crime) - LATE
Reply

treefrog
From: Dunedin, Otago
Since: Apr 2011
Posts: 1115

While not an exact answer, perhaps this may help.

I'm told that the process involves more than 100 judges from around the world, in far flung cities, in far flung countries. It is said that The Boss has the ear and eye of the best film-makers, producers, directors, casting agents, critics and many others of reputable ilk who, under his utterance, watch with avid glee, the offspring of this hallowed competition.

They, the illuminated few, assemble their thoughts and a name or three rises to the surface, perhaps by repetition, perhaps by obvious talent, perhaps by other means, but always obscured from the gaze of mere mortals (but we are assured its kosher.)

These individuals, known personally to (Sir) Ant Timpson, (perhaps not,) genuinely pay attention to the output of this shiny, cloud covered land.

After much debate across the interweb, as you'd imagine in a structure as complex and diverse as the one described, names begin to form a puzzle, pieces of which are hammered into place, forged in fires of argument and amazement, exchanged from solid light and sound into solutions.

The Boss listens, corrals any debate and sorts out the pedant from the bullshit and a winner is selected. And the Masses are appeased.

Well, mostly.
(In truth, 'Chumping' was a bewildering win.)

Hope that 'helps.'

2009-2015 48 Hours Otago/Southland Manager, all-round filmy geek.
2008 - 'Oh! My Gucci Boots!' - Cats of a Feather - Director -*Best Costuming
2007 - 'ARC' - Schrodghers Cat - D.O.P.
2006 - 'Scarred-The Directors Cut' - Strange Attractor - Audience Favourite
Reply

Cardinal
From: Christchurch
Since: Apr 2011
Posts: 182

I hope you're not marked for death now, treefrog.

Seriously though, that sounds (mostly) plausible, but if that's how it works, why not just say so?

Reply

treefrog
From: Dunedin, Otago
Since: Apr 2011
Posts: 1115

in reply to Cardinal:

if that's how it works, why not just say so?

I just did :-)

2009-2015 48 Hours Otago/Southland Manager, all-round filmy geek.
2008 - 'Oh! My Gucci Boots!' - Cats of a Feather - Director -*Best Costuming
2007 - 'ARC' - Schrodghers Cat - D.O.P.
2006 - 'Scarred-The Directors Cut' - Strange Attractor - Audience Favourite
Reply

Ant Timpson
From: New Zealand
Since: Mar 2011
Posts: 340

Fair question.

There is a Judging process of the GF pool to the last lineup of the Grand Final.

All I will say is that it's completely impartial and no bias can intrude.
It is done by people who understand the competition and whom I feel are worthy of making the calls.

The Judges for the GF will be announced shortly.
This International Panel of approx twenty Judges are an unbiased and diverse bunch. They watch the finalists and a winner is chosen.
No one in NZ has anything to do with the selection of the eventual winner.
This was done after I noticed bias in previous judging in NZ (not necessarily in the 48)

All I can say is that is I understand that there is enormous trust placed in this whole process and I want you to know that I take it very seriously. Psychotically so.

For the record Cardinal I have disqualified good friends, banned judges for life and try and keep the winners a surprise. (just try and keep anything secret these days.. it's next to impossible!).

I've thought about releasing the Judges wrap up of the GF so people can where all the shorts place but I feel that's kind of unfair to the teams who place at the end. 12th out of 750 is pretty damn good but 12th out of 13 never feels good.

Facebook - My Facebook Page
Twitter - My Twitter Page
Reply

treefrog
From: Dunedin, Otago
Since: Apr 2011
Posts: 1115

Thanks Boss.

2009-2015 48 Hours Otago/Southland Manager, all-round filmy geek.
2008 - 'Oh! My Gucci Boots!' - Cats of a Feather - Director -*Best Costuming
2007 - 'ARC' - Schrodghers Cat - D.O.P.
2006 - 'Scarred-The Directors Cut' - Strange Attractor - Audience Favourite
Reply

Cardinal
From: Christchurch
Since: Apr 2011
Posts: 182

Thanks Ant. I've got absolutely no doubt that you do your utmost to keep it all as fair as possible and the people we get every year to judge the national final are a great indication of the lengths you're going to in that department, so I take you at your word when you say no bias can intrude.

I can see why some secrecy around I guess what you'd call the regionally sensitive middle part of the judging process that we have now is prudent. I'm still probably going to wonder who's picking the national finalists, though!

Reply

Ant Timpson
From: New Zealand
Since: Mar 2011
Posts: 340

Cardinal. In the old days it was the process was more transparent but all it did was seem to infuriate some and cause others to make accusations about its validity. Bad phone lines and text voting probably didn't help matters ; )

I mean we could open it up, put the finalists pool online and let the public vote for the Final 10, 11 or 12. Yeaaah maybe not.

Let me ask you something instead. What system would you like to see in place? Not being facetious btw, I think it might be the easiest way to answer your initial question by scrutinising what you think is the best way.

Facebook - My Facebook Page
Twitter - My Twitter Page
Reply

dmanthei
From: Christchurch
Since: May 2011
Posts: 418

I appreciate the clarification as well.

As to what system should replace the current one...
from the sounds of it, it's a tough call.

Once films go from regional to national,
even more emotion and investment gets sunk into them,
so I could imagine a seemingly scorned director behaving even more irrationally at that level
than at the comparatively small-potatoes of regional
(which I'm sure gets enough nutters as it is!).

How do you weigh the interests of the good-intentioned national hopefuls
against the potential backlash of allowing on-lookers a window into the process?

Maybe release general descriptions of each of the national finalist judges
so that there's no immediate, easy, low-hanging fruit that the baddies can spam with hate,
but enough information that the hopefuls can be sated and satisfied that justice will prevail?

2014 - Robocrop - "Pads of Passion" RomCom (Best Original Score - Christchurch)
2013 - Robocrop - "The Price of Pilsner" Horror
2012 - Robocrop - "Promises" Crime - DQed (Nominated Best DQed Film)
2011 - Robocrop - "Protocol" One Room (Crime) - LATE
Reply

Cardinal
From: Christchurch
Since: Apr 2011
Posts: 182

in reply to Ant Timpson:

Hmm, good question. I'll say now I don't think public voting should ever come back :)

I guess short of inventing and then running the films through the Judge-O-Tron 4000, there will always be subjectivity involved. There's no getting rid of it - and I think that you can argue there shouldn't be. Look how much everyone loves the idea of the PJ wildcards for example, which are great. Those are just one (legendary) man's (very well respected) opinion, but everyone can get behind them. Therefore I don't think you should be aiming to produce an entirely objective judging panel. I suppose it's like journalism - the ideal is to aim for complete objectivity, while at the same time realising that it's almost impossible for it to exist.

So, I guess if I take that as a given, I don't think it's too big an assumption to make that city judges around the country at least a little bit naturally inclined to be favoured towards films from their own city. I'm not saying that this is due to a conscious bias, a swelling of regional pride, or a sense of favouritism, but just due to being human. These are the films they've seen multiple times and probably developed strong opinions about. This might only be the slightest of effects - because I'm sure that if (when?) any of the regional judges we have were called upon to be judges to decide who national finalists are going to be, they'd all strive to be open-minded and neutral - but I'd be surprised if you could eliminate every last trace of a sort of "that's our baby!" feeling.

As well, whether I'm right or wrong, the sense I've picked up over the years since I've been entering is that there is a smidgen of regional flavour to judging around the country. I think it probably just reflects the different natures of the filmmaking communities in each centre. Again this may just be my perception, and winning city (and national) films are always good no matter which way you look at them, but again if we allow that people are subjective, they're going to have their own particular areas of interest and preferences, and it probably wouldn't be that surprising if we had developed something in the way of slight regional styles both in the making and judging of the films.

So only because of these two sliiiiight factors - which I attribute not to any conscious bias from anyone, but just people being human - I don't think it's ideal (with that goal of as much objectivity as possible in mind) for a significant proportion of a city judging team to also be sitting on any panel / committee that picks who the national finalists are (I don't know if this happens or not - just speaking hypothetically).

I guess there are two ways round that "problem" scenario to get the most objective possible decision on national finalists:

1) Convene a judging panel to pick the national finalists that has the same number of judges from every centre, say 2. That way any of the slight regional feeling that *might* exist is cancelled out by even numbers.

2) Convene a judging panel that has *no-one* that participated in city judging on it at all (like the international panel we have to decide the winner of the final) to decide who the national finalists will be. This way no-one brings anything into the process and everyone watches the finals pool films with fresh eyes.

So immediately you can see significant practicality issues with either of those, but I think either is a good solution in terms of objectivity at least. From what treefrog described above, it sounds like a version of 2 is sort of already happening, so that's great. If either of these two solutions were / are in place though, I'd say just let everyone know that's how it works (still no need to provide scores or ranks or comments or anything), because I don't see on what grounds people could reasonably object to either (although that said, I've seen some of the stuff you've had to put up with previously!)

Reply

treefrog
From: Dunedin, Otago
Since: Apr 2011
Posts: 1115

A potential solution would be a erect a giant dartboard,
with icons of the films within each region of said dartboard,
with the Regional winners closest to the centre, of course.

Select a random, blind hobo, who speaks no English, preferably a foreign national, and give them a sharpened stick.

Spin him/her about, thrice, point them in right direction
and let them heave-ho.
Closest icon to the landing place of the stick, WINS!

Still, I bet there'd be debate about that process too.

But I can say that it has worked well for choosing the site of each Olympic games!

Ants' well phrased, logical and nicely historic document details the immensely difficult process Judging brings.

Any industry that is driven by emotion, ambition, (politics too) and fueled with passion, ego and V is bound to have a vein pulsing in its collective skull! (Please forgive the shambolic metaphor.)

Anyone remember the Music Skirmish of Gisbourne 2007/2008?

Whatever the outcome, it works because it has too.
(I'll reserve my extended litany of wtf for last years win.)

If the currently employed methodology fails us, we must consider a reinvention of the process, until we do have a system that truly functions as an egalitarian, unbiased, arts-centric meritocracy with the broadest possible reach. Good luck and godspeed with that!

Sheeesh. What a predicament to be in.

2009-2015 48 Hours Otago/Southland Manager, all-round filmy geek.
2008 - 'Oh! My Gucci Boots!' - Cats of a Feather - Director -*Best Costuming
2007 - 'ARC' - Schrodghers Cat - D.O.P.
2006 - 'Scarred-The Directors Cut' - Strange Attractor - Audience Favourite
Reply

GloryHunter
From: Queenstown
Since: Jun 2012
Posts: 8

For my 2 cents worth,i think you are getting too complicated,I believe you can use a judging panel from anywhere as long as they know what they're talking about and were not involved in the heats,then it is a simple matter of removing team names and regions from the films ,thus judges will only see genre! much like a wine tasting where a judge scores the wine! : )

Reply

ryan
From: Wellington
Since: Apr 2011
Posts: 237

It's a shame that they have lost the city vs city aspect of the comp.

Was RyanM on the old forum.
Reply

antigraviddy
From: Taranaki
Since: May 2012
Posts: 23

So, winning a regional/city final does not guarantee a place in the National Final?

Reply

dmanthei
From: Christchurch
Since: May 2011
Posts: 418

That's correct.

2014 - Robocrop - "Pads of Passion" RomCom (Best Original Score - Christchurch)
2013 - Robocrop - "The Price of Pilsner" Horror
2012 - Robocrop - "Promises" Crime - DQed (Nominated Best DQed Film)
2011 - Robocrop - "Protocol" One Room (Crime) - LATE
Reply

antigraviddy
From: Taranaki
Since: May 2012
Posts: 23

What is the justification for this? Is the level from particular regions not on a par with other regions?

It just seems odd, using the terms 'heat', 'final' etc.

Reply

Andrew Todd
From: Christchurch
Since: Apr 2011
Posts: 674

in reply to antigraviddy:

Basically you can look at it as the regional competitions being a wholly separate competition (in terms of judging) to the national competition.

The pool of films for the regionals is the films from that region; the pool for the nationals is the films from across the country.

GHOST SHARK 2: URBAN JAWS available for download or stream
Senior Contributor at Birth.Movies.Death

2019 - "Розкол (Schism)" - Science Fiction (National finalist, Chch 3rd place + Best Cinematography, official selection Toronto After Dark and Morbido Fest 2019)
2018 - "Утка (Utka)" - Wish (National finalist, Chch winner + Best Script + Best Animation, official selection Kiev International Short Film Festival 2019 and Abycinitos 2020)
2011-2017 - Christchurch 48HOURS City Manager
2010 - "I Am Single" - Rom-Com (Chch finalist + Best Art Direction)
2009 - "New Fish" - Musical (National runner-up + Best Score, Chch winner)
2008 - "Übermensch" - Horror (Chch finalist)
2007 - "Pain Killer: The Sick Sense" - Superhero (Chch finalist)
ALSO - "As Of Yet Unfinished" (2012, unofficial); "The Contender" (2010, finals intro)
Reply

Cardinal
From: Christchurch
Since: Apr 2011
Posts: 182

in reply to Andrew Todd:

Well, that's not quite true (I assume). Correct me if I'm wrong, but I doubt the people picking the national finalists complete the task of watching every entry from all over the country. Surely the city judges have to be performing some kind of a screening / gatekeeping function in sending on the selection they deem the best from their region (whether it be the top 6 films, all the city finalists, or whatver it is) for consideration for a place in the national final and higher honours. That makes them part of the national judging process.

Reply

themorgan
From: Auckland
Since: Apr 2011
Posts: 655

in reply to Ant Timpson:

I propose that each team is allowed to cast a vote for Popular Favourite. The team would select their favourite (or top three, or whatever) shorts from the screening room. Can't vote for yourself. Then you take the national winner (or top three, or whatever you can fit) of the Popular Vote to the finals.

I think it would quite important that it's not an open vote, but instead requires the team to login to cast the single team vote. Otherwise you're open to the same problems that stupid iPhone video views comp had.

It's also important that we don't see the votes until voting is closed, but I do think those figures should be released after voting closes, and actually winning the popular vote could be a cert itself, so the 3 popular vote finalists know they're in the finals, but don't know how they placed.

This makes for a really transparent selection process for at least these few shorts that make it into the finals.

Frauleins, oh frauleins
2019 | Frank & Beans | Buddy Movie | ULTRA48 | DQ :) | Auckland Winner: Best Disqualified Film
2018 | MUST LOVE BOTS | Anti-RomCom | ULTRA48 | Heat 4 Audience Fav, Auckland Short List, Auckland Finalist: Best Script
2017 | I am ball 2 | Survival | ULTRA48
2016 | The Littlest Punk | Punk | Auckland finalist: Best Original Song
2015 | Crumbs | RomCom, Animated | DQ & Very bad!
2014 | I am ball | Musical, Animated | Auckland finalist: Best Use of Prop
2013 | Crybaby | Race Against Time | Boring!
2012 | S.U.L.A. | Urban Legend | Puppets
2011 | Kill Therapist | Revenge | Writer, Director, Actor | Notorious
2010 | Action Manu | Biopic | Writer, Director | Boring

Fractured Radius
2009 | All's Fair | Musical/Dance | Co-Writer
2008 | The End | Drama | Co-Writer | Auckland winner
2003 | F.I.T. | Cop/Crime | Peripheral
Reply

Cardinal
From: Christchurch
Since: Apr 2011
Posts: 182

in reply to themorgan:

Do you think we could expect much cross-regional judging under such a system though? Again, I dunno if this would be a case of we one-eyed Cantabrians only voting for our own, etc., but more just one of Chch people having just seen more Chch films, Auckland people having seen more Auckland films, and so on - I'd guess that the majority of people involved in the competition see more films at the screening of their city heats and finals than they ever end up watching in the online screening room (certainly in the stage before national finalists are announced, at least).

Reply