Thoughts on anonymized compos
category: general [glöplog]
The point is, that the order is thought to have "some" impact. Probably only a very minimal one, and maybe sometimes even a negative one.
I think most of us would agree here. We are manipulated every day, often even without noticing. So IMO it seems pretty unlikely that from several 100 people voting, none is influenced by the order. One way or the other (makes no difference).
And then, if prods are very close in quality, the final decisive point difference may result from the order.
I think most of us would agree here. We are manipulated every day, often even without noticing. So IMO it seems pretty unlikely that from several 100 people voting, none is influenced by the order. One way or the other (makes no difference).
And then, if prods are very close in quality, the final decisive point difference may result from the order.
Quote:
And then, if prods are very close in quality, the final decisive point difference may result from the order.
If they are equally good, does the final place really matter?
Quote:
The point is, that the order is thought to have "some" impact.
Yes, any order has. The only solution would be to have no order at all, i.e. all watch all demos at the same time?
Quote:
The point is, that the order is thought to have "some" impact. Probably only a very minimal one, and maybe sometimes even a negative one.
I think most of us would agree here. We are manipulated every day, often even without noticing. So IMO it seems pretty unlikely that from several 100 people voting, none is influenced by the order. One way or the other (makes no difference).
There's a lot of "maybe"s in your thought process. And sure, there is a psychological component to the order of a compo I'm sure, but I'm still not convinced it plays a role anywhere to the extent you're implying it does. I gave you two examples contrary to what you're saying. Any concrete data points that demonstrate your point?
Quote:
And then, if prods are very close in quality, the final decisive point difference may result from the order.
Another counter-example to this: Mercury v. CNS at Revision 2016. Mercury was shown first, Conspiracy right after, Mercury scored an extremely narrow win. I also agree with tomkh's point.
For sure this is all hypothetical, that's why there have to be a lot of "maybes". And yes, there has to be an order. So there will always be some bias, that's inevitable.
And, if its not a bias, then the order may play a role very differently too: People getting tired, bored, drunk... or there are technical problems (someone mentioned those in this thread). Whatever you can imagine.
All I am suggesting is, that we should try to minimize that influence (without harming the compo) and secure the organizers from potential (hypothetical) complaints.
And, if its not a bias, then the order may play a role very differently too: People getting tired, bored, drunk... or there are technical problems (someone mentioned those in this thread). Whatever you can imagine.
All I am suggesting is, that we should try to minimize that influence (without harming the compo) and secure the organizers from potential (hypothetical) complaints.
noby: 3 votes from becoming an example instead of a counter-example :)
Random order can not give a less fair vote.
5. doesn't work for demos for quite obvious reasons.
I've only heard this requested once by 1 person, and then it was for music compos specifically. I think anonymous contributions could work in graphics and music compos.
I only care about seeing and hearing the prod on the bigscreen and the reaction, but I definitely get if someone gets a kick out of hearing/seeing their nick announced regardless of what the prod is. Anonymous would ruin that for them.
But yeah, why not? I don't see at all how any unfairness could result from it, except for one thing.
There's the danger of forgetting or mixing up the entries (as always, but easier if anonymous). Hm, mod.hexen_of_valhalla, was that the teenydisco one or the chiptune? If you vote on PC you could have the tunes/pics accessible on each voting page.
Seems to me that in many cases namevoting could be the result from missing the entry or forgot which one it was or exactly what title the entry had. As I see it though, anonymous entries could only result in more fair results, not less.
A less famous musician could cheat (play the system) by giving his entry a name that will make people think a famous musician made it, for example. So you'd have to vet the titles. The alternative of numbered entries is doomed, too many would get the entry numbers wrong.
Obviously, someone could walk up to the guys he wants to namevote for, just like now, and ask for their entry numbers or titles. Or just ask someone, "which was the spacesynth one, I loved it!". So.
Best way I think would be to poll musicians and artists going to the party what they prefer and go by that, then ask them after if it was better.
Random order can not give a less fair vote.
Quote:
So this is something that I've been mulling over for years, and last weekend I had a long discussion about this with Offwhite before/during the graphics compo: I personally think hiding the author names during music/graphics compos is a net loss.
Here's my throught process on the matter:
1. The supposed main purpose of doing this is to avoid namevoting, which I can understand, but then I thought more closely and here's what I ended up with:
1a. People who want to win by their name only will find a way to get the entry->name association delivered; they can just tell people which one was theirs, either before or after the compo; with the Internet they don't even have to do it in person. For the record, I haven't seen much of this happen.
1b. There is a syndrome what I would call indirect or passive namevoting, where the author doesn't want to be voted on based on their name, but they're popular enough that people do it for them anyway. That being said, this often happens because those specific authors have a distinctive style (h0ffman, Made, Prince, etc.) which will come through whether the name is shown or not.
2. It's technically implausible to keep the names from coming out; the authors are allowed to post the entries after the compo on Soundcloud/Facebook/etc, and there's of course the whole party FTP issue which Revision overcompensated for by converting down everything, but to the point where there's no ID3s or song titles left so the whole compo is just one big blob of indistinct files, which feels like throwing the baby out with the bathwater. (I'm sure the actual files will show up sooner or later, but as it stands it feels like a nuclear option.)
3. I only realized this at Revision, but I've seen and heard a few entries which I really dug to the point that I would've liked to discuss them with the authors during the party, but couldn't because of this. It dampens a lot of the social aspect - just hanging out with some of the authors after compo and talking about the compo is one of the most fun parts.
4. On a corollary to that, why is this so specific to graphics and music compos? Is there demonstratably no namevoting in demos? It's just feels oddly specific and arbitrary to me that some compos would be prone to it and others wouldn't, and the argument that "names are easier to remove from a picture" just doesn't feel right - you could ban credits in demos too, but in those cases it's part of the tradition so we don't? It feels inconsistent.
5. Another corollary to point 3 is that a big part of the competitor experience (for me at least, but I'm sure for others too), is seeing your name on the bigscreen - for newer people it's the thrill of jumping in the pool, for veterans it's the reaction of the silence or applause that kicks in just for that. Granted, I can see how this might be part of the aforementioned bias, but I would argue that when the crowd goes fidgety when e.g. h0ffman's name shows up on screen, it's because he earned it.
I'm well aware that the presence of actual namevoting is incredibly hard to quantify, but to me anonymous compos just feel like an overcompensation to something that may not even be a problem in the first place.
Thoughts?
5. doesn't work for demos for quite obvious reasons.
I've only heard this requested once by 1 person, and then it was for music compos specifically. I think anonymous contributions could work in graphics and music compos.
I only care about seeing and hearing the prod on the bigscreen and the reaction, but I definitely get if someone gets a kick out of hearing/seeing their nick announced regardless of what the prod is. Anonymous would ruin that for them.
But yeah, why not? I don't see at all how any unfairness could result from it, except for one thing.
There's the danger of forgetting or mixing up the entries (as always, but easier if anonymous). Hm, mod.hexen_of_valhalla, was that the teenydisco one or the chiptune? If you vote on PC you could have the tunes/pics accessible on each voting page.
Seems to me that in many cases namevoting could be the result from missing the entry or forgot which one it was or exactly what title the entry had. As I see it though, anonymous entries could only result in more fair results, not less.
A less famous musician could cheat (play the system) by giving his entry a name that will make people think a famous musician made it, for example. So you'd have to vet the titles. The alternative of numbered entries is doomed, too many would get the entry numbers wrong.
Obviously, someone could walk up to the guys he wants to namevote for, just like now, and ask for their entry numbers or titles. Or just ask someone, "which was the spacesynth one, I loved it!". So.
Best way I think would be to poll musicians and artists going to the party what they prefer and go by that, then ask them after if it was better.
Quote:
And, if its not a bias, then the order may play a role very differently too: People getting tired, bored, drunk... or there are technical problems (someone mentioned those in this thread). Whatever you can imagine.
Like the soundsystem breaking and fucking up the first few entries - I've been there. Our intro was played second (not second to last) out of #10, because ASM uses that randomization you prefer :)
Quote:
Best way I think would be to poll musicians and artists going to the party what they prefer and go by that, then ask them after if it was better.
Or, to have a public discussion about preferences and ideas, you know, like this thread.
Quote:
...because ASM uses that randomization you prefer :)
That's clearly NOT the kind of randomization I prefer. :)
Then what is? What makes you think that e.g. the last few entries in the 64k compo at Revision 2016 were for example not randomized? And if they were randomized, why do you dispute the order they were randomized to?
I started by asking exactly that, if the orders where randomized in some way and only got an answer from las (see page 4), stating that he does not randomize. I can only guess what was done for the other compos at Revision.
And I do NOT dispute the order of any past compo at all. I am discussing about the future here.
But the discussion so far pretty clearly shows me, that besides me probably nobody sees a problem waiting for an answer here. So I better stop now.
For me, it was interesting and instructive. :)
And I do NOT dispute the order of any past compo at all. I am discussing about the future here.
But the discussion so far pretty clearly shows me, that besides me probably nobody sees a problem waiting for an answer here. So I better stop now.
For me, it was interesting and instructive. :)
Quote:
noby: 3 votes from becoming an example instead of a counter-example :)
Not really. The margin of being so tiny that would still imply that two prods virtually as good as one another are unaffected by one being shown as the last entry. Now had Conspiracy won by say 20% more votes, that would be significant.
Guys, it's neither example or counter-example, unless you have more knowledge, i.e. if those entries are equally good (<=> no bias) or Mercury's intro is significantly better, but because of the "bias" scored almost equally.
However, we could easily get this knowledge, looking at pouet voting ;)
At a time of writing this:
Mercury's intro: 152 thumbs up, 3 piggies, 1 down
Conspiracy's: 160 thumbs up, 12 piggies, 0 down
This supports the assumption that both entries were in fact equally good (or Conspiracy even slightly better, also note that both get similar attention), which subsequently supports the argument that there was no bias towards preselection order. Unless of course, there is a different kind of bias on pouet ^^
However, we could easily get this knowledge, looking at pouet voting ;)
At a time of writing this:
Mercury's intro: 152 thumbs up, 3 piggies, 1 down
Conspiracy's: 160 thumbs up, 12 piggies, 0 down
This supports the assumption that both entries were in fact equally good (or Conspiracy even slightly better, also note that both get similar attention), which subsequently supports the argument that there was no bias towards preselection order. Unless of course, there is a different kind of bias on pouet ^^
See, THAT'S namevoting! :D
Is it just me, or aren't cdc's a lot more worth than thumbs?
Statistically? No, CDCs don't count in any sort of stat.
Morally, well, up to you I guess.
Morally, well, up to you I guess.
@tomkh: Pouet votes are not independent of the position the entry got in the compo (for one, you have different populations that look at only top entries vs. all entries), so it's useless as a measure of “was the compo placement correct”.
The only way we could solve this would be to divide a demoparty (with a large enough population to get meaningful statistical results) randomly into two groups, shown the same compo with different order, and see if the results compared equal or not. And then probably repeat the experiment several times, because there's going to be funny intra-group effects in the mix, too. (Also, you could try putting each scener into a separate room with no communication with anyone else before they voted. I'm sure they would be thrilled at the experience!)
I don't think this will happen until we have a reasonably precise way of simulating human brains, and at that point, Evilbot will have taken over anyways.
The only way we could solve this would be to divide a demoparty (with a large enough population to get meaningful statistical results) randomly into two groups, shown the same compo with different order, and see if the results compared equal or not. And then probably repeat the experiment several times, because there's going to be funny intra-group effects in the mix, too. (Also, you could try putting each scener into a separate room with no communication with anyone else before they voted. I'm sure they would be thrilled at the experience!)
I don't think this will happen until we have a reasonably precise way of simulating human brains, and at that point, Evilbot will have taken over anyways.
I´m pretty sure random did not have any influence on the compo order ;)
Sesse: you are fully right, those are dependent observations. This "additional knowledge" would only help if pouet users would ignore the position of the entry in the compo or vote in favor of the entry that got 1st place, but even if it looks like a valid assumption to me, we really don't know.
The red heart-like icon greatly affects the perceived value of a cdc. I suggest changing the icon to a kidney or liver or football. Let's see how the cdc distribution changes over time! So you have not taken into account the FONTS and GRAPHICS that are presented on voting forms. Aha! More things to speculate over.
I thought about changing it to a flower but ever since Undertale flowers can fuck right off.
Bump! Revision is coming up. Would be nice to hear something from the orgas on this I think.
Good point.
the order in compos should be picked by Paul the Octopus to guarantee absolute objectivity!
All demos should be time-stretched to exactly one minute length, so the length wouldn't affect votes, and the demo measurement devices (i.e. "voters") would give more accurate readings.
The ordering problem is important. If there are, say, 10 entries, then there should be an exact integer multiple of 10! voters. Each entry ordering permutation is shown in an otherwise identical hall, simultaneously, to a group of voters, each group carefully selected to have a similar distribution of lamers and elites. Because it might be challenging to have millions of identical halls and audiences, or even more than one actually, the maximum number of entries shown should be limited to one.
The ordering problem is important. If there are, say, 10 entries, then there should be an exact integer multiple of 10! voters. Each entry ordering permutation is shown in an otherwise identical hall, simultaneously, to a group of voters, each group carefully selected to have a similar distribution of lamers and elites. Because it might be challenging to have millions of identical halls and audiences, or even more than one actually, the maximum number of entries shown should be limited to one.