Staffa Staffa

Devaluation of old scores

▼

Devaluation of old scores

from Stardock Forums
To keep the Metaverse filled with active players rather
then clogging the top ranks with old farts who dont
play anymore, old scores are devalued over time.

I heartily agree with the intent but not the method.

I dont want to go into why the method currently used
isnt that good, CM has done a fine job of that. But unlike
CM I do not merely wish to adjust the numbers to mask the
wierdness that occurs with his specific set of data, cause
that wierdness occurs regardless of how its masked in a
single instance.

When you want to remove old games from the metaverse
in a slow, continious fasion, you do so by giving less
weight to the older games.

To take an example. I want to average two numbers, lets
say 1,000 and 10,000. But I want to give 5 times as much
weight to the 1,000 number. This means that the 1,000
number is 5 times more important then the 10,000 number.

I would first multiple the 1,000 number by 5, to get 5,000
then add the 10,000 to get 15,0000, and then I would divide
by 6 to get to 2500. This is the same thing as saying I
averaged 1000, 1000, 1000, 1000, 1000 and 10000. I just
weighted the 1000 5 times as heavily.

Now, using 20% devaluation per month as an example(though
5% would be more realistic)

Scores that are a month old would be worth 800 points but
add only .8 to the square root divider, 2 months and 600 and
.6, etc until its zeroed out, essentially removing the
score competely from consideration. Which at 20% would
happen after 5 months.

Some numbers to show what would happen

This example assumes you submit 1, 1000 point game
once a week.

With no devaluation:
1000, 1414, 1732, 2000, 2236, 2449, 2646, 2828, 3000, 3162, 3317, 3464, 3606, 3742, 3873

With 20% devaluation
1000, 1414, 1732, 2000, 2191, 2324, 2408, 2449, 2449, 2449, 2449, 2449, 2449, 2449, 2449

Notice that after 2 months the score competely stabilizes.

Thats because your entering new(identical) scores as fast
as they are being removed.

In a more realistic setting, so long as you individual
scores improve, your overall scores will ALWAYS rise. In
fact, if your earlier scores are much lower then your
more current scores(as is often the case), your score
will rise even faster as those earlier games are taken
out of consideration.

This basicily represents a method to score only your more
recently played games, as its equally effects everyone
there is no unfairness in it. Playing more games will
still be advantagous and so long as you remain active
you will never be penalized by having your individually
scored games becoming worth less over time.

To sum up, this basicily makes your more current games
more important, it does not make your older games of less
value, contrary to what some might think, there is a
difference.

800 points when adding a .8 to the divisor is still valued
correctly just not as heavily. Unlike adding 800 points
while adding 1 to the divisor, which is what is currently
being done.

Actually im not that good at math, techinically you might
have to subtract the sqrt of .2 from the divisor each month
rather then .2. I did the math both ways and the end result
is essentially the same.


1,329 views 57 replies
Reply #26 from Stardock Forums Top
CRAP!

Now I feel like Stardock ;) :lol:

When I added the enhancement of using a more real world set of data, I introduced a typo bug into my function to decay to 0 (dang global replace while forgetting to select 'whole words only'). As a result, in the graph up there in post #19, old scores are being dropped much too quickly from the data set.

The 10% decay rate is fine, see this graph with the bug fixed:
First, real world data set: http://filebox.vt.edu/~channum/metaverse/18months_wildtype.gif
Second: constant data set: http://filebox.vt.edu/~channum/metaverse/18months_constant.gif

At any rate, sorry for the screw up there. If you really want to see a 5% per month weight change, I can do it, but the decay rate is going to be so much slower, it's going to take close to 4 years to run a full cycle.

The gist is that your method does run a bit higher than my proposed fix, but not so much that it's worth fighting for one over the other because the curves shapes remain the same, their only major difference is what happens after you stop submitting.
Reply #27 from Stardock Forums Top
And I repeat, I would MUCH rather fall competely
off the metaverse then remain at some vastly
diminished level of my former glory.
End of quote


It's not so vast, though. The longer you keep at it, the more of your peak value you retain after you quit. Your method guarantees you disappear, while the decay with a basment method (or even the decreased weight with a basment) means that you could conceivably stay on top forever, you'd just have to have no life and play the game for the next two years :HOT:
Sins of a Solar Empire: Rebellion
Reply #29 from Stardock Forums Top
Staffa, the weighted average going to 0 is the red line. It is 10% per month as, like I said, 5% is too slow. You won't have the first score drop off until month 20 of playing. When I get home, I'll run it just to show you, but it's going to look more like the no decay at all graph than any of the others.
Reply #30 from Stardock Forums Top
Oh I know it wll take 20 months before a score drops off
competely :)

5%*20, not hard

Of course they will start going down much sooner then
that, just slower over time.

Anyway, 10% doesnt look bad, better then that wierd
arse dip both the non controls take :)

Reply #31 from Stardock Forums Top
Btw, mine does dip too, but only after the player
stops playing very often, yours dip even while they
are still very active, thats the wierdness :)

Anyway, your probably right, my way is better but
it is more then a 2 character bit change in the code,
its at least a few bytes.

Reply #32 from Stardock Forums Top
And 5% would smooth it out even better, and I wont
start editing posts again until they fix edit!
Reply #33 from Stardock Forums Top
There's no dip in the decay method I'm using by virtue of it's method, that's simply an effect of the data set, that's why I posted the constant data input graphs as well. Although you can't tell what significant effect (if any) a change will have in everyday type situations, you can't compare the methods without a fixed reference point, either.

Yours, fwiw, experiences that same dip, but it's masked by the leveling out of the overal score as well. The decay method actually does start to climb again which makes it look slightly more pronounced.

Although, to play devil's advocate for a minute, since I've been coding and recoding various changes to the scoring algorithm over the past 2 days, if they had a fixed algorithm to implement, it wouldn't take much effort to do any of the methods we've been looking at and playing with.
[Message Edited]
Reply #34 from Stardock Forums Top
The major difference, as i see it, is that given consistent input (which may not be very reasonable as an assumption) that Staffa's method will reach an assymptote and stay there indefinitely, whereas CM's method will continue to increase.

I for one would not want to see a situation where you had to keep playing just to tread water, I think that would get old in a hurry. But, it is difficult to really get a handle on real-world performance here.

CM, have you guys been playing long enough to use real data, perhaps pooled across a couple of people? (that may be an utter waste, given scoring changes)...just a thought.
Reply #35 from Stardock Forums Top
Your two both peak at week 33 and then, looking only
at the mini peaks, continue to dip down until week
54.

Mine peaks at 49, and then goes down at the exact
same time that the peaks start spreading out a lot. Ie
when the player starts playing less frequently.

Mine also remains extremely level during this less
playing period, and then dies away competetly when
the player is done(as it should imo)

If my score isnt my highest score, regardless of
how little it does drop, id just assume it drop off
totally when im done.

Reply #36 from Stardock Forums Top
Ok, CM, here is the biggest difference.

Take your set of data and add a year break of no
games and then add some more data sets at about the
same level as when they quit.

That might be interesting.

Reply #39 from Stardock Forums Top
I implemented the 5% step weight decay to 0 as per your request (and it's waaay to lenient unless you happen to believe that people will be playing this game for 3 years each).

I will try the break because I have a sneaky suspicion that your method is much harsher because while it lets you play without the "dead weight", you're starting from ground 0. For somebody coming back to a game after a year, having to fight as hard as they did as a newbie just to show up on the metaverse doesn't sound like much fun.

I need to fix that real world data set anyhoo. The quick and dirty way I cooked up to generate an arbitrarily large data set has some issues with the way I used %mod values in the formula. It doesn't actually don't slow down submissions as much towards the end as it should as a result.

At any rate, that's probably about as much as I'm going to do for this weekend, this is starting to become work ;)
Reply #40 from Stardock Forums Top
You guys have too much time on your hands ;)
Reply #43 from Stardock Forums Top
Why don't you guys ask Stardock to release the raw Metaverse data, and you can compute your own rankings?
Reply #44 from Stardock Forums Top
Why don't you guys ask Stardock to release the raw Metaverse data, and you can compute your own rankings?
End of quote


Our own rankings aren't the focus. I discovered a flaw in their current decay schedule that, as Bearcave put it above, put you in a position of treading water (in fact, it pretty much guaranteed you would drown).

Then, as is likely to happen, me and Staffa wound up on opposite sides of how to fix this glitch ;p
Reply #45 from Stardock Forums Top
I have another proposal how to fix the problem. Let s1, s2,...sN ordered (from best to worse) list of scores with any type of aging applied. I propose calculate the overall score in the following way:
calculate score in usual way only for 1 game this will be s1, for 2 games (s1+s2)/sqrt(2), etc., and take as overall score the max of those.

Advanteges:
1. With this method of calculating submitting the game will never decrease the overall score, so more games will be submitted.
2. Old games will not punish active player, starting from some moment they just will not be included in overall score.
3. Any new scoring method as transition from 1.04 to 1.05 will not hurt active player overall score, comparing to new players.
Reply #46 from Stardock Forums Top
Actually, Sergey, your method has the very flaw that the current method avoids: there should be no 'perfect set' of scores and your system allows that in spades. Any system that doesn't "punish" you for older scores allows you to submit 4 20K games and then sit on your butt forever.

To be open to new gamers you *must* "punish" people for old scores. Whether you do it through time based decay of the value or the weighted value, it still needs to be done.
Reply #47 from Stardock Forums Top
I think you missed something. If you submit 4 20K games your score will be 40K, but if you submit 9 20K games the score will be 60K, so there is no possibility for perfect set.
Reply #48 from Stardock Forums Top
No, didn't miss it. That was just a quick example of why your system doens't work. So fine, 9 20K games = 60K, in 10 years, it's still worth 60K, and that is why your idea won't work given what Stardock wants from the metaverse.

I know I've wasted more than enough of my own time trying to improve the scoring system, but I'm doing it in the spirit of what Stardock has done (and therefore somewhat likely to implement). I'm reasonably certain that they're not going to adopt any system that doesn't have time based decay of your overall score as part of it (and given some of Brad's grumpier comments this past week, they're not going to adopt anything that requires more than a few lines of code to change no matter how good it is as well).
Reply #49 from Stardock Forums Top
Again it seems that you missed that my system didn't assume that there is no aging, Calculation of max should be done after applying aging decreasing of the scores.
Reply #50 from Stardock Forums Top
I for one prefer some sort of weighting formula to score degradation. Why should a score go down simply because it is old? That would be like this. "Hey there Sanders. You know that World record of yours? It aged. IT's no longer a world record." When in fact no one had beat it. The fair thing to do is weight it less in the calculation.