Site scoring system (Feb 2nd 2013)

Information of the Rating system.

Re: New site scoring system (Feb 2nd 2013)

Postby sinnybee » 28 Jan 2013, 09:20

super_dipsy wrote:
sinnybee wrote:I'd guess that your descriptions will likely better apply to 1400-1800 and over 1800 for the site.

I honestly don't know how things will turn out, but in the modeling and tuning I paid quite a lot of attention to the history of the site. I realize you can't claim this is an accurate comparison because as has been said before, if you knew the scoring rules in place you might have played differently and chosen your games differently. But it is the best historic data to hand and there is a LOT of it (the old 'never mind the quality, feel the width' argument ;) ). After all, we are almost at 25,000 completed ranked games on the site now.

If I just pretend the old data run through the new system has some value,

Woah! You ran all the old data of ranked games chronilogically through the new formula?
I wish I had a back-stage-pass for this.

super_dipsy wrote:then if we just stick to players who have played 5 games or more (so discounting people who come in, try a game or two and don't like it and leave, which tend to overwhelm the 1000 point area of the curve :) ) then if you were >2000 you would be in the top 0.5% of the site. If you achieved 1500-2000 you would be in the top 6% of the site. So I am reasonably comfortable on that data that
• Ratings in the 1500-2000 region can be considered to be very respectable. Ratings of over 2000 are likely to put you clearly in the top level of players on the site

has some merit. I would say top half a percent is clearly in the top level of players, and top 6% is not at odds with a rating that is very respectable. The reason I chose the numbers I did was I was mentally think something along the lines of top=0.5% and very respectable=5%, but the numbers were nice and round at the 6% mark. However, your statement is no less true, since the wording of 'top' and 'very respectable' is so imprecise.

So in the groupings you chose, the old data shows >1800 = 1.5% and 1400-1800 = 8.3%. I would be just as comfortable calling the first group the top players, and the second the very respectable ones. However as I said, the old data is by no means an indication of what will happen. We wait and see - exciting isn't it :D

Wow... numbers.
It is exciting.

I wish the two long PMs I wrote to you these past few days hadn't got accidently deleted (no longer logged in while not ctrl-C copying my text in time and then accidentally pressing F5 :( ).

I really wish I could have been talking about all this stuff weeks ago, though I was around in recent months talking about using a elo-type system for the site.

I do still have some worries, which I will hopefully get a chance to type up in a PM again.

I have seen the often occurance of thousands of online players dropping pending games because the average score of the opponents in the game was lower than that of their own (or of the average score of their team/partnership).
It happens oh so often, because the games have an elo-based formula with an algorithm of 400 which doesn't suit the game. I like games with a lot of strategy like Euchre and Diplomacy, but Chess has even more skill involved and more of a skill spread.
At the gaming site I played at the most, the algorithm of 400 was used, in that if the difference in score of the player(s) and that of their opponent(s) was 400 or more, the game was automatically unranked, and if the difference in score was almost 400 and the underdog(s) won, their score would recieve a HUGE boost.
However, though Chess has many different tiers of skill level in which an algorithm of 400 or even 350 could be used, an algorithm of about 500 or 550 may be more appropriate for games like Euchre and Diplomacy (with an even higher number that may best fit games with a lower amount of strategy involved in the game).
So, as I was saying, at different online gaming sites and in different games, there would be the excessive habit of a large number of players of not wanting to play against those ranked lower than them. This was because the algorithm of 400 overestimated the difference in skill between those of a set difference in score.

I'm in huge support of the change of our scoring system, but I would prefer us to have an algorithm that would best avoid players dropping pending games.

...And these sites allowed players who created games to set a score range allowed for others to join with (like their own score give or take 50, 100, 150, ...300 points), which is a fantastic feature to have, but it still doesn't remove the problem I'm speaking of.
Gold Classicist since 1-11-11
FT Asst GM of 35 player WWIV Aug 2011-Feb 2012
#1 ranked player of playdip early 2013
4th highest forum karma count at Apr 2013 ending (behind Craw, Dipsy, and Rick)
Tournament Director of the 31 game PDVT Feb-Dec 2014, the first playdip tourney with over 100 sign-ups
User avatar
sinnybee
 
Posts: 5816
Joined: 03 Sep 2010, 07:01
Class: Ambassador
Standard rating: (1332)
All-game rating: (1467)
Timezone: GMT-7

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 28 Jan 2013, 11:21

I think there are one or two things that might help us avoid completely falling into the hole that other sites sometimes fall into with their rating systems. For a start, there is the ratings shield. This will hopefully encourage players to step in and take up surrendered positions, regardless of the standard of opposition. Remember that if your shield is active you cannot lose,and it does not even consume a fading echoes slot because there isn't any fading echoes :) . The other is I anticipate mor Anonymous games, where you can't see who is playing anyway.
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby asudevil » 28 Jan 2013, 14:26

I get your point sinny...but how many people are that obsessed with rank...

Although the anon thing only being for premiums will keep plenty of non-anonymous games available.
Captain FANG, forum team championships WINNER
Part of the surviving nations of WW4/Haven

Unless I am in the cheater's subforum. 99% of what I say is NOT as a mod.
User avatar
asudevil
Premium Member
 
Posts: 16582
Joined: 18 Jul 2011, 02:20
Class: Star Ambassador
Standard rating: 1351
All-game rating: 1447
Timezone: GMT-7

Re: New site scoring system (Feb 2nd 2013)

Postby gannymede » 28 Jan 2013, 18:04

I see a potential for a big spike in the number of surrenders. As I understand the new scoring system, a surrender just counts as a loss. That will encourage a lot of players to surrender as soon as they think they don't have a chance of winning, whereas before they would have played on just to avoid the one point loss. Most of us have enough pride not to do so, but I can imagine that a lot of people will just throw up their hands and quit as soon as things start to go badly for them. Especially people with a three-game limit.

I know you are not ready to re-open the rulebook and I hope I am pointing out something you have already considered, but I am a bit worried.
Homo sapiens non urinat in ventum
User avatar
gannymede
Premium Member
 
Posts: 97
Joined: 18 Jan 2012, 04:16
Location: Toronto
Class: Star Ambassador
Standard rating: 988
All-game rating: 2007
Timezone: GMT-5

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 28 Jan 2013, 18:15

gannymede wrote:I see a potential for a big spike in the number of surrenders.

it is something we plan to monitor. I am not at all convinced that there will be a spike. For that to be true, it would have to be true that the -1 in the old system was preventing a lot of people from surrendering. I am not sure that is true. I think that people who are going to surrender would do so regardless of a -1 on their score. People who surrender do not care about a) their score and b) their reliability. Most players I think are far more put off from surrendering by having it marked up against them as a surrender, demonstrating they are unreliable. I know numerous players who will not had passwords out to protected games to players who have a history of surrenders.

But if you are right, then we have the ability to change it. If we do have to change it, I think I might prefer to go with the carrot vs stick approach - so perhaps give players who play the game through a +1 rather than -1 for surrendering.

We will see, and act accordingly :)
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby super_dipsy » 28 Jan 2013, 18:19

I hit enter too fast. I meant to mention a couple of other things too. I suspect people who care about ratings may make a lot of use of the ratings shield, so they can have 'free plays' where they are at no risk of losing any points. But remember that surrendering discharges your ratings shield immediately. Also, in the old system you used to get people dropping into a game in place of a surrendered country (+1) and then if they don't like what they see they will just themselves surrender (-1). But if they try that in the new system, they will get clobbered with the loss. So that also may prove a dampener on surrenders, as well as those extremely irritating situations where someone drops out and someone comes in, you start to talk, then they drop and someone else comes in, etc etc etc
User avatar
super_dipsy
Premium Member
 
Posts: 12068
Joined: 04 Nov 2009, 17:43
Class: Ambassador
Standard rating: 1000
All-game rating: 931
Timezone: GMT

Re: New site scoring system (Feb 2nd 2013)

Postby gannymede » 29 Jan 2013, 06:49

That sounds pretty reasonable. Especially since you say you might modify the surrender penalty if you see a problem. I'm looking forward to starting with the new system.
Homo sapiens non urinat in ventum
User avatar
gannymede
Premium Member
 
Posts: 97
Joined: 18 Jan 2012, 04:16
Location: Toronto
Class: Star Ambassador
Standard rating: 988
All-game rating: 2007
Timezone: GMT-5

Re: New site scoring system (Feb 2nd 2013)

Postby WarSmith » 30 Jan 2013, 05:40

Cool. Have been waiting for an excuse to come back ;)
"A lie gets halfway around the world before the truth even has a chance to put its pants on”
User avatar
WarSmith
Premium Member
 
Posts: 2105
Joined: 01 Mar 2009, 22:12
Location: Scandinavia
Class: Ambassador
Standard rating: 1724
All-game rating: 1716
Timezone: GMT+2

Re: New site scoring system (Feb 2nd 2013)

Postby TheProfessional » 30 Jan 2013, 08:15

I'm really glad the new system rewards solos over draws, and calculates them as being worth more than double a two-way draw. However, I wish solos were even more blatantly rewarded. I am tired of playing games where as soon as you've got three players dominating the board, there are talks of a three-way draw. Draws are a way of ending the game when there is basically no possibility of anyone gaining a solo, but at the moment many players see them as an end in themselves (albeit a less worthy end than a solo). This usually means that a player who is aiming for a solo has less than 1/7 chance of getting a solo, because there are a lot of players who are happy to end in a draw and will gang up on anyone who looks like they might be after a solo. I hope the new system fixes this to some degree, but I wonder whether enough has been done. (Perhaps not everyone agrees that it is something that needs fixing).

I have some more specific questions relating to solos and draw scoring.

Regarding the potential of losing points from a draw, does this depend on the ratings of all the players in the game, or just those remaining in the draw? For example, if you have a game with a few high rated players, and a few low rated players, could the high rated players eliminate the low rated players and then avoid losing points from drawing because they're pretty much equal? Conversely, could low rated players gang-up on the high-rated players, and then end with a draw that lifts their score because they managed to eliminate the high-rated players?

More generally, does the new system encourage low rated or high rated players to team up against one another in any way? If not, how is it avoided? I think it would be quite bad if there is any meta-gaming incentive to team up with this player over that player.

This has come up before, but I haven't seen an answer. Is it possible to lose points from soloing (I am guessing not, but confirmation would be nice).

Can you give us some idea of how much better a solo is compared to a two-way draw (without telling us the algorithm obviously). For example, if you have evenly rated players in a game, how much better would winning the game be compared to a two-way draw?

Is it possible to get a high rating without ever getting a solo? Relatedly, how do players who frequently draw, but never solo, compare to players who rarely draw, but do better than 1/7 for solos? For example, who is the rating system likely to favour: a player who draws 50% of the time, but never solos, or a player who solos 20% of the time, but never draws (holding fixed the ratings of all the players).

If you feel like there is one answer that sort of addresses all of these together, I'd be happy with that :)
TheProfessional
 
Posts: 24
Joined: 30 Dec 2010, 01:32
Class: Diplomat
Standard rating: (978)
All-game rating: (977)
Timezone: GMT+10

Re: New site scoring system (Feb 2nd 2013)

Postby BlueBorjigin » 30 Jan 2013, 09:03

super_dipsy wrote:A solo win can never lower your score.
User avatar
BlueBorjigin
 
Posts: 222
Joined: 21 Oct 2011, 21:28
Location: Toronto
Class: Star Ambassador
Standard rating: (1160)
All-game rating: (1177)
Timezone: GMT-5

PreviousNext

Return to Site Scoring System

Who is online

Users browsing this forum: No registered users and 3 guests