Devaluation of old scores

from Stardock Forums
To keep the Metaverse filled with active players rather
then clogging the top ranks with old farts who dont
play anymore, old scores are devalued over time.

I heartily agree with the intent but not the method.

I dont want to go into why the method currently used
isnt that good, CM has done a fine job of that. But unlike
CM I do not merely wish to adjust the numbers to mask the
wierdness that occurs with his specific set of data, cause
that wierdness occurs regardless of how its masked in a
single instance.

When you want to remove old games from the metaverse
in a slow, continious fasion, you do so by giving less
weight to the older games.

To take an example. I want to average two numbers, lets
say 1,000 and 10,000. But I want to give 5 times as much
weight to the 1,000 number. This means that the 1,000
number is 5 times more important then the 10,000 number.

I would first multiple the 1,000 number by 5, to get 5,000
then add the 10,000 to get 15,0000, and then I would divide
by 6 to get to 2500. This is the same thing as saying I
averaged 1000, 1000, 1000, 1000, 1000 and 10000. I just
weighted the 1000 5 times as heavily.

Now, using 20% devaluation per month as an example(though
5% would be more realistic)

Scores that are a month old would be worth 800 points but
add only .8 to the square root divider, 2 months and 600 and
.6, etc until its zeroed out, essentially removing the
score competely from consideration. Which at 20% would
happen after 5 months.

Some numbers to show what would happen

This example assumes you submit 1, 1000 point game
once a week.

With no devaluation:
1000, 1414, 1732, 2000, 2236, 2449, 2646, 2828, 3000, 3162, 3317, 3464, 3606, 3742, 3873

With 20% devaluation
1000, 1414, 1732, 2000, 2191, 2324, 2408, 2449, 2449, 2449, 2449, 2449, 2449, 2449, 2449

Notice that after 2 months the score competely stabilizes.

Thats because your entering new(identical) scores as fast
as they are being removed.

In a more realistic setting, so long as you individual
scores improve, your overall scores will ALWAYS rise. In
fact, if your earlier scores are much lower then your
more current scores(as is often the case), your score
will rise even faster as those earlier games are taken
out of consideration.

This basicily represents a method to score only your more
recently played games, as its equally effects everyone
there is no unfairness in it. Playing more games will
still be advantagous and so long as you remain active
you will never be penalized by having your individually
scored games becoming worth less over time.

To sum up, this basicily makes your more current games
more important, it does not make your older games of less
value, contrary to what some might think, there is a
difference.

800 points when adding a .8 to the divisor is still valued
correctly just not as heavily. Unlike adding 800 points
while adding 1 to the divisor, which is what is currently
being done.

Actually im not that good at math, techinically you might
have to subtract the sqrt of .2 from the divisor each month
rather then .2. I did the math both ways and the end result
is essentially the same.


1,340 views 57 replies
Reply #1 from Stardock Forums Top
Please note again, 20% is an example, its also rather
fast in my opinion, I think 5% would be better
Reply #2 from Stardock Forums Top
Staffa,



You know, I didn't realize it before, but if I understand the current method correctly (devalue numerator with same denominator) and you use a similar scenario (one game per week with constant score), the metaverse score will increase forever...it does NOT bottom out.



Take an example of one game played per month with a score of 1000. In this example, the score is devalued each month by 20% until it drops to 40% (just for you CM :D )



As the number of games gets big the formula for the score approaches:

.4*1000*sqrt(number of games) + 1000/sqrt(number of games +1)



the .4 is for the 40% factor

1000 is the same score you used before



Look carefully at the second term which reflects how much the score changes each month. As n becomes big, this term approaches zero. Therefore, in an asymptotic sense the forumla for the score at month n (where n is big) reduces to:



.4*1000*sqrt(number of games) or the more generic

Deflation*Score*sqrt(n)



Where Deflation is the percentage of the score kept regardless of how long ago it was played. (note that if this were 0, then your score would eventually approach zero...but Brad has more sense than that :d ) and Score is the score of each game (still assuming one game per month)



It does not matter how many months you play in the metaverse at a constant rate (i.e. how big n gets), your score will continue to increase, albeit by a small factor relative to your score.



Again, I'm not making any judgement or saying this is good or bad, just trying to help explore the issue.



If I screwed up some math, kindly point it out.

Edit: actually, I think the second term would have .4*1000 in the numerator, but the conclusion is unaffected.
[Message Edited]
Siege of Centauri
Reply #3 from Stardock Forums Top
Actually, that's the point of both mine and Staffa's approaches. Your score under either system will reach a point where the rate of increase is minimal, it will continue to increase but much more slowly than before. For instance, with a 40% basement, it increases rapidly for about a year, then hits a point where every new score is tempered by every out to pasture score -> after 2 years of playing at the same rate, you'll only see a 25% increase over where you were at 1 year.

Staffa's system works similarly, I just haven't bothered crunching the numbers to see what its curve is.

If Brad does go with a 50% basement like he implied, then you'll level off at about a year, but will increase at a somewhat more rapid rate from that point out.

The potential difference between using my quick and dirty mod of the existing system, and Staffa's weighted "average" system is that Staffa's with enough tweaking could more accurately reflect recent performance.
Reply #4 from Stardock Forums Top
Actually, assuming CM did the math right, cause I
didnt actually check him. He said that it peaks
at 10 months, then goes down for 2 years
and then heads off into the wide infinite(though limited) open.

Of course this assumes identical games played consitantly
CM wants to change the numbers but keep the method.

My posistion is that the system is flawed and the
numbers he wants to use just mask that flaw with the
data set he is using to test it.
Reply #5 from Stardock Forums Top
I wasnt clear, but the first description is CM's
explanation of the 'problem'

His solution is to change the numbers so it basicily
does what your example said it would do with that
specific set of data.


Reply #6 from Stardock Forums Top
It's independent of the data set actually. All I did was assume a player who submitted games with a fixed frequency and a given average score, beyond that, the overall curves are identical no matter what frequency and what average score is used, all that changes is the peaks and the magnitude of the slope.

If I get bored tonight, I'll generate a nice line graph of the curve(s).
Reply #7 from Stardock Forums Top
Check out robuk!

As of right now:

1 game, simple/huge

10504 points

rank 16
Reply #8 from Stardock Forums Top
wow fsk+, what has that to do with devaluation of old
scores? :)

I guess the seperation between different difficulty
levels will never be fixed
Reply #9 from Stardock Forums Top
I'm gonna repost something I did in another thread here, since it's relevant:

I'm going to try and help out Staffa's explanation some. The current formula is for the scores to be added up and divided by the square root of the number of games. I'm going to show the results of a few proposed changes based on 2 (very contrived) examples.

Player A bought GalCiv way-back-when, played 50 1000-point game, and quit. Then he dusts it off and plays another 50 1000-point games in 2 days.

Player B just got the game and has played 100 1000-point games in the last few days.

Now, both players are equally skilled, and under the current system recieve the same score, 100,000/sqrt(100) = 10,000. However, we'd like to make it so Player A's old games don't count as much, without totally ruining his score.

Suggestion A: Make player A's old games disappear. Now he has a score of 50,000/sqrt(50) = 6509.4, and could easily make up the difference by playing another 50 1000-point games. Not a terrible choice.

Suggestion B: Make player A's old games 500-point games (50% value). Now he is at 75,000/sqrt(100) = 7500. A higher score than with suggestion A, but to make up the difference, if he played 50 more games, he'd end up with a score of 10206...but it would be MUCH harder for him to improve that score than the it would for the new guy to improve his.

Suggestion C: Make player A's old games 500-point HALF-games (50% WEIGHT). Now he is at (50,000/2 + 50,000)/sqrt(50 + 50/2) = 8660. With only 25 more games on his part he'd be on an EXACTLY level playing field with the new guy...this seems to be the best option to me.

The difference between devaluation and weighting is the difference between B and C...and you can see that 50% VALUE and 50% WEIGHT have very different results.

I'm not terribly concerned either way, but thought I'd try and clear the waters a bit.

-EtherMage


~SDC~
Reply #11 from Stardock Forums Top
I'm not terribly concerned either way, but thought I'd try and clear the waters a bit.

-EtherMage
-----
I'm with you there...makes little difference to me..just trying to help. I think you've created a very simple yet powerful example. nice work.
Reply #12 from Stardock Forums Top
OK, since a graph is easier to understand, here are some metaverse progression curves for some hypothetical player with each of the potential decay basements illustrated (20% is the current amount, 50% is what Brad implied he might alter things to today). While I appreciate the power of the weighted average systems proposed by Staffa and Ethermage, the truth is you're going to get the 2 line change to the current algorithm if you get anything at all. I think these curves illustrate nicely that my "kludge" fix is every bit as useful as the more complicated ones.

For each of the curves, this player is assumed to have submitted games for 4 years and then stopped (the curve goes through the 5th year showing the final decay).

Graph 1: Player submits 1000 point games every 5 days: http://filebox.vt.edu/~channum/metaverse/1K_x_5days.gif
Graph 2: Player submits 5000 point games every 2 days: http://filebox.vt.edu/~channum/metaverse/5K_x_2days.gif
Graph 3: Player submits 10000 point games every 5 days: http://filebox.vt.edu/~channum/metaverse/10K_x_5days.gif
Graph 4: Player submits 5000 point games every 20 days: http://filebox.vt.edu/~channum/metaverse/5K_x_20days.gif
Reply #13 from Stardock Forums Top
Another graph which shows why Staffa's idea is no different than what we have now: http://filebox.vt.edu/~channum/metaverse/staffa.gif

To keep it in context, I made the decrease in weight 10% per month for 10 months (this is the same period that the current method and implied fix decays over).

The only differences between Staffa's proposed 'weight to 0' weighted system and the existing system is it peaks and levels off slightly higher, doesn't have the slow decline and slow rise because it just goes completely level, and you drop completely off the face of the metaverse 10 months after you stop submitting.
Reply #14 from Stardock Forums Top
For the final irony of ironies, if we were to fix Staffa's "true" weighted system versus that abominable decay method by not lowering the weight to 0 but rather let it level off at, say, 0.2, we get the following: http://filebox.vt.edu/~channum/metaverse/fixedstaffa.gif

Yep, the difference between my "flawed and fixed for a specific data set" and a fixed version Staffa's methods are nigh undetectable. And the difference between Staffa's original proposed fix and the current method are negligible.



[Message Edited]
Reply #15 from Stardock Forums Top
Im tired but will address this further tomorrow, though
the point is purely acedemic.

I just wanted to say that I would much rather fall
off the face of the metaverse competetly then be
left at 20% of my peak. Better to be removed then to
be player number 32,301

Reply #16 from Stardock Forums Top
Though if you want to save me the trouble of number
crunching and do some more graphs, as you seem to
enjoy it more so then I.

Do some datasets that are more typical, where a
player at first improves signifigantly and then
continiously gradually improves, assume the metaverse
scoring system is static and we dont have the bizare
score inflation.
Reply #17 from Stardock Forums Top
I'm planning on doing that (need to rewrite the function that generates scores) just to show a more typical curve (including decreasing frequency and quitting before 4 years are up, that period is an artifact of how long it took the current system to normalize).

This is barely any effort, well, barely any effort for an anal retentive geek when it comes to analyzing game mechanics ;)

I wrote a program to run this simulations and export the data in a comma delimited file which I then just open in Excel.
Reply #19 from Stardock Forums Top
OK, last simulation unless anyone besides me is enjoying this and has some sort of good request.

This is a more "real life" set of curves. The player starts off submitting every 2 days at 500 points. The score increases quickly at first, then more gradually (peaking out at 7750 points. The player's interest gradually dwindles until only 1 game every 20 days is being submitted. After 1 year, the player stops submitting altogether: http://filebox.vt.edu/~channum/metaverse/reallife.gif
Reply #20 from Stardock Forums Top
Funny thing is that in a more real life scenario, a pure version of your method blows chunks imo, my method of raising the basement holds up better, but the fixed weighted average method I tossed out as a modification to yours actually seems to work best, interesting.
Reply #21 from Stardock Forums Top
CM,

I don't understand where the dip at the end of the original graphs is coming from. Oh, duh, now I see it. That's the 5th year decay.
Reply #22 from Stardock Forums Top
Is that using 20% or 5%?

The decay rate is a lot faster then I would have
thought using 5%.

I did say that 20% is just too quickly show how the
data would change. In actuality I would use 5%.


[Message Edited]
Reply #23 from Stardock Forums Top
It's using the same relative decay as the current method -> 10 months to obsolescence, therefore 10% a month. I can play with the numbers, but the curve shapes will remain the same since when scores gets pulled down remains fixed.

A weighted 'average' system like you want is going to need to have some basement to make it palatable to the mainstream player. I've played around with both 10 & 20 & find that in scenario where your scores are gradually increasing that there's not much difference; the trick is to have some residual value for older scores, though.

I'm still futzing with this because tweaking the program is entertaining me. I want to run a set of simulations where the player remains active for 18 months because that is closer to what we're likely to see from the main metaverse players (that, and differences in trends will show up better than ending it at 12 months when all of the methods are roughly the same).
Reply #24 from Stardock Forums Top
My method is not the same, 5% is a better amount.

Really, this is perhaps more idealogical then anything,
but old games are not a of lesser skill value then
newer games. Just dropping the score without dropping
the denominator smacks of mathematical foul play.

The same thing can be acomplished using a more balanced
approach.

You will notice when you 'fixed' my method, it grew
quite a bit faster for quite a bit longer then any
of your methods. Simply because it removed old, low
scoring games from being counted at all.

The problem with my non fixed method is that its not
my method, I said 5%, thats not 5%.

lets see the graph with 5 % :)
Reply #25 from Stardock Forums Top
And I repeat, I would MUCH rather fall competely
off the metaverse then remain at some vastly
diminished level of my former glory.