Quest Suggestions and Constructive Critiques
There has been a lot of carping and bellyaching here, from time to time, about the way this contest is constructed or has been conducted. I think it would be a better notion to start a forum topic where we could give CONSTRUCTIVE, POSITIVE comments to PRX for future improvements.
One could argue that this suggestion is premature, given that we are still only in the middle of round two. Perhaps that's true. On the other hand, I'd rather be too soon than too late with such ideas, considering that participation seems to be waning as time goes on.
So, I'll start:
1) Too Few 2nd rounders
I believe that the initial cut from 1,400 to 10 was far too drastic. Many more interesting initial entrants deserved a second round. I also think a broader 2nd round would hold public interest in the contest longer - simply more people to root for. Finally, I think we could have had a much stronger field of 10 "third-rounders" after a 2nd round of 40 or 50 entrants. That would be more exciting.
2) Over-conservative Judging
Many of us seem to believe that the judges' selections were simply too much like what's already being heard over NPR, except for the welcome addition of some younger/ethnic voices in this contest. But how about some entrants with more of a twist?
3) Over-emphasis on Verbiage vs. Edgier Ideas
For my money (admittedly, a total of $0.00), the contest - at least so far - has emphasized verbal skills over point-of-view and smarts. Certainly, a lot of hipper/riskier ideas heard in round one were summarily dropped in seeming favor of a more routine "hostiness."
****
That's my humble contribution!
Rich Meitin
http://www.publicradioquest.com/node/1038
delicious
diggI might suggest that you take a closer look at the voting. While you will give a 5 rating to their favorite, it is also true that a 1-point rating for their competition will, mathematically, will have a much greater effect on your favorite’s ranking. If one contestant got one vote with a 5 rating and another contestant got 100 votes, 99 with a 5 rating and one with a 1 rating, the contestant with the one vote would have the higher average rating.
Possible solutions (and, I will admit, none is perfect);
1) for any voter who gave only one 5 rating and all other votes were 1 ratings, still count the 5 rating vote, but eliminate the 1 rating votes.
2) Eliminate all 1 rating votes
Regardless of what contestants average ratings are, you should be sure that the contestant(s) with the most 5 ratings is promoted to the next round
Not knowing how sophisticated your vote tallying system is, you may or may not be able to do any of these.
I'd love to see prtq2 use instant runoff voting (commonly known as IRV) or Condorcet voting.
These types of voting systems eliminate the "spoiler effect", famously illustrated by Ralph Nader's 2000 campaign. Both IRV and Condorcet allow folks to clearly articulate their preferences, allowing them to vote for their long-shot candidate without fearing that doing so would result in a de facto vote for a candidate they loathe.
While prtq may not be as high stakes as the presidential election, and may not be so prone to "spoiler" issues, it would still be a great place to get people familiar with these excellent yet somewhat unknown alternative voting methods.
just how many of the 15000 registered voters actually voted?? I have no choice but to feel less than 200 people voted altogether....with a majority of them being 11 pointers(or 22 pointers) if they came back for round two to support their hero.I wished we could see an actual number...I also feel[as many of us did] in round one..that 11 pointer votes should be disqualified...because the ones who already have a large fan base prior to this submission will automatically have a higher # of votes,plus a higher # of 5 point ratings...which is totaly bias and not based on sole talent.
I, for one, don't think all the stink about voting we have seen is worth the discussion. First, the vox pop vote only affected ONE of ten places on the roster.
Second - there is NO voting system that cannot be subverted. Want to rule out 11 pointers? Fine. But you're going to have to rule out 22 pointers, then 33 pointers, etc. There's always a work-around.
Third - if voting were the be-all and end-all, then Tattoo Todd would have been the 10th finalist, by a WIDE margin. But he wasn't.
The easiest way to avoid poisoning the 10th position via voter stacking is to give the PRX judges a silent veto power. Which I believe they have, anyway, if I'm not mistaken.
Just my 2 cents.
Rich Meitin
www.richmeitin.com
http://www.publicradioquest.com/node/1038
and raise you a penny Rich! yes your right..only ONE person can advance to the next round that way...yes only ONE person can win that spot on popular vote...but do you honestly think it to be fair that one can beat out another who may be more talented or have more potential than another based on a larger fan base coming in and voting for them? yes, the judges may have veto power, but do they use it? will they use it? how do you determine weather or not that person derserves the spot over the others?? hince my suggestion of eleminating the 11/22 pointers! that way you take out of the equation the fan base, and leave nothing but the legitimate users[the ones who listen/vote for all entries] thats all I'm saying. I'm not looking to take away anyones 15 minutes of fame...but isnt the whole process to determine a potential host for a national show? you wouldnt want someone to get that based on pre-exsistant fan bases would you? I know I dont...I figure someone can win this based on talent and skill alone...totaly with out the help of 300 pre-exsistant fans...at least thats my prespective. so all in all it seems of little meaning to you that just one little person can advance that way, but I find it to be a huge deal if they do it by that means...because they're some finalists here that deserve a better shot than the others and the judges can only pick 6 of them.....so let the talent speak out loud NOT the pre-exsistant fan base!
but that's just me...well acually thats alot of peoples feelings....are you still in disagreement?
Hey Chad -
I get your irritation, and I understand that with you, it's a matter of principle. But your proposed solution doesn't really overcome my objections:
1) Look at the ACTUAL result - clearly, the judges exercised a veto over Tattoo Todd, right? How could it be otherwise? He had HUNDREDS and HUNDREDS of 11 pointers. Yet, he didn't make the finalist list. So, the judges' veto IS in force, and that SHOULD take care of your objection, at least in the case of lesser-talents with a big fan base. (I don't mean here that Todd was a lesser talent. He sounded quite good to me, albeit perhaps more fit for AM or FM. My point is only that the veto works.)
2) If you make a new rule to drop 11 pointers automatically, it's still absurdly easy to get around that new rule. You just get your fan base to do 22 points worth. Or 33. Or 44. Where do you want to stop? Do you also want a rule about people who do fifty 1-stars and one 5 star? There are lots of ways to subvert any voting system. (The Electoral College system is notoriously twisted. Even simple "Majority Rule" has terrible side effects: it disenfranchises large minorities completely. Ask any statistician. ALL voting systems can be picked apart or abused.)
3) Frankly, if the contest is 90% fair and only 10% skewed - possibly (see point one) - that's not too bad, in my book. Again, the judges aren't stupid. They aren't going to allow any outright abuse.
Moreover, unlike you, I see an existing fan base as one measure of talent. For example, people enjoy Todd and he had a following, which is one measure of talent, yes? If PRTQ wanted to rule out professionals, they could have done so. But they considered and rejected this idea. (Todd - sorry to keep using you as my example. I mean no criticism at all.)
I don't see how your current arguments deflate these objections. Perhaps we should just agree to disagree. I appreciate your passion and principle. Maybe I'm TOO pragmatic or accepting.
I do like the Instant Runoff voting idea mentioned here, quite a lot. Maybe that's a better, more ideal way to go. I'd like to ask a math person if it could be subverted by a large fan base.
Rich Meitin
www.richmeitin.com
http://www.publicradioquest.com/node/1038
And I agree with you about fan bases. However, what about people who vote one person a 5 and the rest with 1s? Since this is the first year, I'll give it a pass because no one would know how it would turn out. Since we know the system, if you want to vote, then vote for everyone and have a ranking between 1-10. So, one gets a 10, the second person they like gets a 9, and so forth. That is not a perfect system as well, but it works best.
thats a huge flaw...in the fact that it's possible for someone to vote for their favorite then turn around and pass out ones to the rest of the contestants in a vile attempt to lower the others chances of advancement....BUT, luckily for us, the people I refer to as "11"pointers, are to ignorant or lazy to realize that, and therefore you dont see a lot of that happening, because if they did, then they will have 101 points total, showing they voted 10 times and commented once....but I'm sure that happens none the less, but like I said it doesnt appear that it happens allot, but it is something I hope the judges watch out for....people do spiteful things sometimes, cant possibly eleminate all possibilities.....
I think we have to start having some faith in the system -- because there's some things that can be done in regards to some points raised and I'd think they've already been done. 'Popular votes' -- the judges are clearly watching and can see whether there's an imbalance. It really would be easy for a fan on or with access to a College campus to just book time on a computer server and use multiple entries.
It would show up soon enough that someone was coming from a particular campus in the vicinity, or if they were from a workplace that had more than one computer -- like a software sales center or a general lab. But that would have been thought of before and the nature of the internet would lead to that conclusion and taken into account. In the end the judges do have final call!
Round Three could pose problems if one of the people chosen to be interviewed was a cult figure (has anyone interviewed Steve Jobs, yet? Hundreds of iPod fans log onto vote for the sharp-thinking finalist who interviewed their hero!) and they came into vote for that contestant on the idea that perhaps they'll get them onto public radio too. So I'd suggest that this sort of thing is taken into consideration, if it hasn't already. A particular political spin can have people voting for the interviewee rather than the interviewer.
It's up to all of us to focus as we should on the quality and not the quantity of the votes in that regard -- and be realistic about how the final sifting will result in a final three who reflect talent and acumen - rather than a fictionally logistic power base of canny internet-access tactics. There's been hundreds of computer competitions before and phone-in competitions and this isn't like the film "Quiz Show"! It's up to the true talent in the end and the judges are the best to call on that. Let them do that job and encourage all of the entries to show their quality!
You are correct. Good post.
Rich Meitin
www.richmeitin.com
http://www.publicradioquest.com/node/1038
Thanks for these comments, we're reading every word here at PRX, even when we don't have time to weigh in.
There are lots of lessons learned, and a number of the points on this thread are ones we've discussed at length at PRX and on the Talent Quest team.
One major restriction on the way we've been running the project is that it's all done through a grant from the Corporation for Public Broadcasting (CPB). When you get a grant from CPB you have to sign a contract that outlines the plans and "deliverables" of the project in considerable detail, including the budget for various expenses (in this case, award money to finalists, hosting costs for the site, etc).
There's very little flexibility as the project gets underway, and if you want to make changes you have to file an amendment to the contract and potential go through additional negotiations with CPB.
Needless to say this limits much in the way of improvisation or reacting to events. Particularly since we submitted the original proposal and started the contract months and months before the contest actually started.
Back then we weren't at all sure how well this thing would go, and in our wildest dreams we didn't think that nearly 1500 people would submit entries in Round 1 (there's even a part of our CPB agreement that covers what we should do in the event that fewer than 10 qualified entrants materialize!).
So I totally agree it would have been great to bring forward a much bigger swath of the Round 1 entrants. And there are a bunch of other ideas we're still exploring about ways to create opportunities for more of the talented folks from Round 1, most of which we'll only get to when we get a tiny bit of breathing room (in addition to running the Quest our small team in Cambridge is keeping PRX and a bunch of other related projects in motion).
It's not clear that there will be another PRTQ, or if there is that we'll have anything to do with it, but we can still imagine together ways to do it better or differently if there is a next time. But please also keep the suggestions coming for things we might be able to do now while we're still in the midst of this one!
- Jake
It's interesting to learn how obligations to CPB restrict your range of motion now that this thing is in motion.
Good to have you weigh in, Jake. Judging by the hour of your post, you're burning it at both ends.
To be clear - our friends at CPB deserves tremendous credit for coming with the idea in the first place, putting up some real money to make it all work, and having the risk-taking vision to encourage PRX to follow through on what must have initially seemed like a crazy contest idea.
One thing we hoped for but ended up not having time or capacity to pursue was to line up a corporate sponsor who might have kicked some more unrestricted money we could spend on various wish list items for the contest, the site, the prizes, etc. I suppose it's not too late, so if anyone reading this has an uncle who is VP of Marketing for, like, Sony or Apple please drop me a line ;)
Of course they have most of the podcasts. You could try small with SplashCast. Or, you could ask Facebook, especially Mark Zuckerberg.
Or Grey Goose.
hindering your ability to adapt to emerging realities. Pity.
If there is ever something resembling a PRTQII, I might suggest switching the activities of Rounds 1 and 2. Round 1 could take care of (a fairly rapid) initial screening for minimum capabilities where everyone is on a level playing field. For those who make that grade, cut them loose and see what they can do. It might make Round 1 more boring; but apples to apples and a larger group of those with a first-person stake in it should carry it through.
Thanks again for doing this. It has given a number of us another chance to believe in ourselves even if few others lend credence to that view.
If they're were going to be a PRTQ 2, the field will expand dramatically and how can they keep track. I would like, if it's possible, have each local affiliate have their contest for best talent, then have the local winners go to a nationwide competition for a chance to have their show be nationally broadcast. That would take a lot of cooperation from stations, but it would be worth it.
I'd be delighted if this thread is of at least a little help to you folks. Also, it's fascinating to hear about this from YOUR side of the microphone!
Keep up the good work.
Rich Meitin
www.richmeitin.com
http://www.publicradioquest.com/node/1038
Thanks Jake for the insightful comments. I totally forgot that this is funded by CPB and understand why there are restrictions and a small timeframe. I would be also shock that you had almost 1500 entries that initially would be a small group of people and this turn out to be exceeding your own goal for this concept. I would love it if it return next year and hopefully there are amendments to this so the contest will be more compelling. I know this was a tough job, but you, the judges, and the crew have done an excellent job on setting up the contest the first time around. Continue the good work.
I agree with much of what was said. Here's what I said in another thread re. my ideas about how to do it differently next time:
"I think the range of talent that got through to round two - from very good to not - provides a very strong argument for having many more people get to round two and simplifying the time/energy needed on the PRX side for the second round next time - e.g. have 100 people progress, have each one call in to an automated system to get a random impromptu word and create an automatic 2 min recording, ditto with a shorter read, plus let each person upload the billboard" (using the same system as they bdid in round one).
Deborah
Listen away at: http://www.publicradioquest.com/node/703
If I could offer what I feel would be the strongest advice it would be … TIMING TIMING TIMING. The momentum was lost when round one ended and it was three weeks before round 2 was posted. The camaraderie and excitement of the boards and the conversations of the possibilities of this contest came to an end the longer we waited.
There was a suggestion made on another blog that the same word be given to all of the contestants. I agree. It would have been far more interesting – and revealing – to hear the finalists expound on the same word or phrase.
Yes, I was disappointed to see many individuals whom I thought were quite outstanding not be part of the finals. But this is part of any contest, and I appreciate the scope of what the judges took on in order to round down these entries. It would have been quite a bit more enjoyable, I think, to have more choices after the first round. I believe it would have served the needs of the contest better as well.
On the positive side, I have appreciated the whole concept of the contest and the incredible scope and versatility of the website. I am grateful for the openness & assistance of the judges and appreciate the breadth of the task they have taken on to judge so many entries. I have also been impressed with the range of talent that has come together on this site.
Overall, I agree Rich 100%. 1451 to 10 is a big reduction. At least have a bigger pool. Also, if I bring 5 random people giving their top 10, you would have 50 different people. The other two parts I think were in part of the system. I use to work in public radio and these judges know what they're looking for. If there was a pyramid format of 1400 to 100 to 50 and down to the final 10, that would of been more exciting.
I will say still support the top ten because they do have talent and hey, they might surprise us later on down the road and also this is wonderful support for public radio. Everyone wins.
We should certainly support our favorite finalists, and support PRX - there's a long way to go, yet.
Rich Meitin
www.richmeitin.com
http://www.publicradioquest.com/node/1038
I agree with each of those points. I admit it must have been hard to know what would happen, but that cut from 1400 to 10 was a KILLER, cutting this site down to nearly nothing. In this second round, it suddenly seems like the contestants have all of their extended friends and family voting, there isn't quite the community that once seemed to be on here (I hardly recognize anybody's name!). And I DEFinitely feel that a lot of MY favorite, and certainly many of the edgier/more interesting ideas were tossed off in favor of more traditional Public Radio voices; I was hoping this contest had as its remit the possibility of expanding Public Radio listeners into the new millennium, rather than just finding new people who do the same thing. Shows what I know! Ha!
Still engaging, I suppose. Like our government, I can't look away!
"It's all in the mind, you know.” - Spike Milligan
Recent comments
2 days 7 hours ago
3 days 7 hours ago
1 week 3 days ago
2 weeks 2 days ago
3 weeks 5 days ago