Some thoughts on the Metaverse

from Stardock Forums

The Metaverse has a couple thousand scores on it. About 5% of the people who have purchased Galactic Civilizations have used it so far. And about .05% of them are very focused and concerned over the robustness of the system.

The Metaverse was designed for 3 major reasons:

#1 So that Stardock could look at the top players strategies and incorporate their strategies into AI updates. We have been doing this and you will see much of this in the free expansion pack. But we have been doing this in 1.01, 1.02, and 1.03 as well. That is one of the big reasons why the AI plays more intelligently.

#2 It allows players to band together to create empires and in general form an on-line community.

#3 It allows casual players a persistent high score system in which they win medals, move up in ranks, get awards, etc.

Metaverse scores age. That is, games played in previous months do lose some of their value. This way, it gives people who buy the game later a chance to do well and keeps things fresh. But more importantly, it helps us avoid the dreaded "reset" that hard core players tend to want to clean things up. Resets are bad because they adversely punish casual players who just think it's cool to be ranked at all and they are a temporary solution anyway. Aging has the same effect without punishing casual players.

But still, the players most focused on their metaverse scores have shown increasing concern over potential abuses in the system. Most GalCiv players are adults. And so to a certain extent, the honor system is in place. That said, it is statistically certain that a percentage of people will abuse the system. Hence, people who are really into the metaverse have been asking that we put more effort into "plugging the holes".

The problem with that is that like a strategy game, game development is about managing limited resources. Any engineering hour devoted to making the metaverse even more robust is an engineering hour not devoted to improving the game itself.

Let me give you an idea of some of the stuff that would be involved in this, because we have given this issue thought for a long while:

1) You would need to do CRC checks on approved mods on the library vs what the user has when they submit a game. If they are using a non-approved mod then their score is heavily penalized depending on the values of that mod.

2) You would need to implement CRC checking on saved games themselves so that someone couldn't take a hex editor or something to their data.

3) You would need to have an encrypted submission system so that someone with a packet sniffer couldn't modify the data as it was being transmitted.

4) You would need to recreate the game such that anyone who wants to submit scores to the metaverse without being heavily penalized plays under very specific conditions (i.e. everything above plus certain map sizes, no Ctrl-N, limited saved games, more emphasis on making sure starting positions were somewhat consistent).

These 4 things don't represent the full requirements to have a truly robust Metaverse. They represent the beginning of what would need to be done.

What would the result be?

a) The casual users, the people who make up 90% of the metaverse players, would demand that there be a localized high score system. And rightly so.

b) It would ruin the casual player's experience for just wanting to be ranked, somewhere, on the system, getting new ranks, medals, etc. because they would have to play the more restricted settings to get their score to count.

c) Players would have to make the false choice between playing the GAME itself for fun or playing the metaverse (i.e. having to choose the restricted options to get their metaverse score to count).

And on top of all that I've mentioned, you would have the never ending battle of someone finding some loophole in the AI that no normal player would do but would give them some sort of scoring advantage on the Metaverse.

Now, some of this stuff we can budget into a Galactic Civilizations II someday in the future. But the effort required here is as much development time as we have budgeted for the Expansion pack. Which would you prefer? Which do you think most users would prefer?

I've been part of ladders on the net since there have been ladders. I can tell you that no matter what you do, there will always be a percentage of very vocal people ticked off at the system.

The current system isn't terrible. It's not broken. Like most ladders, the top 1% or so of players are probably prone to really focusing on their scores with the other 99% just casually interested. Such is the case on all ladders. A heavily modded game and a non modde dgame might make the difference of a couple thousand points. Some cheese tactic or whatever might make a couple thousand points too.

But ultimately, the system, as a whole, works pretty well. If you're a good player and spend enough time playing, you can get to the top of the scoring system. And many of them are no longer playing GalCiv anyway, they're playing progress quest with the Metaverse being the object of the game ratehr than the game itself. 

Such players are doing no harm to anyone. The 99% casual player group could care less. And that 1% player group will include people who will be testing the boundaries of how much they can get away with.

1,244 views 45 replies
Reply #1 from Stardock Forums Top
Brad,

I respect your decision and support it.

Will also be supporting our modders by downloading their mods from the Library.

Now that all this is done with, get us our AI enhancements and new features! :p
Reply #2 from Stardock Forums Top
I can tell you that no matter what you do, there will always be a percentage of very vocal people ticked off at the system.
End of quote

Well, I've been moding a forum with 3000+ users for over a year and made the same experience: No matter what you do, there will be those who blame you for being to tough, and those who blame you for not being tough enough, and those who blame you for - being.
After a while of mostly frutiless discussions, I came to the conclussion that there is only one thing you can do: Decide what course to take, explain it once and then it's take it or leave it.
In my personal opinion, this is what you should do: Don't waste time following the arguments that go on in this forum wether PQ should be rated in the scoring this or that way... it leads to nothing... you change it, someone else complains.
And the amount of work needed to make the Metaverse secure against cheating is ridicolous compared to what would be gained.
About 5% of the people who have purchased Galactic Civilizations have used it so far.
End of quote

That means: 95% are playing just for their own fun and don't care about metaverse. As I am. The only reason for me to submit to metaverse is
#1 So that Stardock could look at the top players strategies and incorporate their strategies into AI updates.
End of quote

Because to do something like that is an awesome cool thing and part of what sets stardock apart from other developers. And I would be very disappointet if stardock less or no time left for that because they are trying to satisfy the 0.05% of costumers who are complainig about the metaverse scoring system.

So, frogboy, I agree with your opinion and hope that you're statement that there won't be put much work into the scoring system, but into the GAME, will be understood and exceptet as well as the reasons for that.
And, if some people shouldn't: Don't bother. ;)

PS: Hope my english is okay.
Siege of Centauri
Reply #3 from Stardock Forums Top
Given the choice about having the Galciv team spending most of its time policing the Metaverse or coding the game and getting AI ideas from the Metaverse data, I think I will take the game development over the Metaverse. I would rather have a better game, thank you. It would be nice if anything that is a major exploit and can quickly and simply handled be fixed, but us players are not in any position to know what exactly comprises simple or quick when we are not the ones who coded the thing.

So what we really got here is a matter of trust on a couple of levels. Trust that the Galciv team will at least look at a major Metaverse problem and fix it if it is indeed quick and simple, but the same trust that if they do not fix it there must be a good reason behind leaving it be. (Yes, that means accepting that Big Brother may know what's best, at least in Galciv.)

The other level is trust on a player to player level, as in honor and a reputation for fair play. If we encourage that as a player community, that is what we will mostly get, a majority who cause no problems with cheese-of-the-day. If we players adopt a "Wild West" attitude in that whatever works works as long as it pumps the scores, then we will be known for that. In essence, we need to maintain a community built upon the common ground that we play Galciv and play Galciv indirectly with each other through the Metaverse as opposed to a loose band of cheesers who play the system. And since we are playing each other, we should respect each other and the rules, and not looking at how to exploit the rules at whatever it costs our fellow Galciv players.

What does this mean to a modded game? If you heavily mod Galciv, then do not submit a score using those mods. The library mods should be like sanctioned house rules that are OK to use. And a hacked game is of course always illegal. Will some people try to submit anyway even if they are using a non-regulation mod or a hacked file? Of course some will. But we will just have to discourage such abuses in order to preserve our community's reputation.

Remember, it truly is a two-way street. The Galciv team needs us and needs to be able to trust us to handle ourselves in a responsible manner, and we need the Galciv team and that team needs to be trusted to listen to us and handle the game in a reasonable manner.
Reply #4 from Stardock Forums Top
Frogboy: "c) Players would have to make the false choice between playing the GAME itself for fun or playing the metaverse (i.e. having to choose the restricted options to get their metaverse score to count)."

But this is already the case. If I want my scores to count, I can´t play on small maps, can´t play on maps with non-abundant habitable planets, and have to avoid fast victories, because they give me fewer points (less pop, fewer techs).
Reply #5 from Stardock Forums Top
The other level is trust on a player to player level, as in honor and a reputation for fair play. If we encourage that as a player community, that is what we will mostly get, a majority who cause no problems with cheese-of-the-day
End of quote


This echos my sentiments from the previous thread - treat the Metaverse as an HONOUR system, and let Stardock get on with making a better game, rather than bashing their heads against the insoluble problem of creating a ranking system that is completely bombproof.

Coffeedragon: I know you feel you're not being rewarded for effort, and we discussed this on your own previously very interesting thread. I think your issue here is how 'honest' scores are WEIGHTED, rather than the different issue of hacked/modded games - I could see some justification for better rewards for small map scores, for instance. Staffa's thread on PQ versus population as a 'quality' indicator is another example. I think those issues apply equally to single player game scoring as much as the Metaverse and that is different to Frogboy's issue - and certainly one that deserves attention. For example, I think some players are frustrated that they play small maps (usually due to personal time constraints) on high difficulty levels, but are poorly rewarded compared to playing huge maps at simple difficulty.
Reply #6 from Stardock Forums Top
Further game development should be priority #1. The bonus pack looks very promising and I rather would like to have it in June than for instance in September if this would be caused by deviding limited time to be spend on other things than game development. :HOT:
Reply #7 from Stardock Forums Top
"I think your issue here is how 'honest' scores are WEIGHTED, rather than the different issue of hacked/modded games..."

I think these issues are related, though. If not even outright cheating is given much attention, then, I assume, the issue of score weighting appears to be even further down the priority list (namely, not a priority at all). :(
Reply #8 from Stardock Forums Top
There were changes made in the scoring in 1.04 (which I don't have yet), and I think one of the issues addressed was the scoring-vs.-mapsize one.
Reply #9 from Stardock Forums Top
This is just my opinion;

1. I love mod's its one of the main features of the game it keep the game new and fresh.

2. I agree with frog boy on changing the metaverse it would smiply be to much work to revamp as stated and works fine for people who love mods.

3. I believe you will never stop cheaters or hakers

4. I would LOVE a SEPERAT METAVERSE board for NON MODERS. :CONGRAT:

5. still my favorite game. :)
Reply #10 from Stardock Forums Top
May you at least use my suggestion on player score calculation so it will never be punished for sending a low game scores? It's fair and simple to implement.

~SDC~
Reply #11 from Stardock Forums Top
I don't care about others abusing or scoremilking. Like Coffeedragon i just want the scores to reflect the effort a bit more. I don't think i've seen Frogboy mentioning this issue. And i'd like to know if it's an issue at all.

Some questions for Frogboy:
- Why do techwins score most points?
- Is it intentional that a gigantic simple win scores more points than a small masochistic?
Reply #12 from Stardock Forums Top
I see you have finally removed my 23k score game, are
you now going to remove every game where someone
inflated their population?

That would be almost every game above 6k. Thats the
problem, its a creeping tendency that you cant avoid.

Im not asking you to do anything but stop punishing
people for submitting intersting/lowscoring games
and fixing the redicious way in which population
is used to determine score. If people want to cheat
the system, like you said, its not worth worrying
about. But the problems that prevent the system from
being used correctly(even if it is just by 1% of the
population) should be fixed.

Btw, I saved my 23k score right before I inflated it
I dont suppose I can resubmit it with a more socially
acceptable score? I mean after all, thats what it all
boils down to, currently 8-14k is socially acceptable.
23k is not, nor did I intend it to be, that was the
point.
Reply #13 from Stardock Forums Top
I agree that the metaverse should not be the focus of your efforts, but I disagree that you should just take a "screw it" approach to adjusting some of the more glaring problems it has currently. If you never had any intent of it becoming a scoring ladder, you should have not have had score involved at all and merely awarded various ranks and medals for submitting increasing numbers of won games on various difficulties/map conditions. Such a system would have had people purely playing the game and not the metaverse, and if that was your intent, you failed miserably with the given system. You made it part of the game, it deserves some amount of attention.

Ideas such as the one for using total PQ under your control versus population is one such thing that could be used to fix "running up the score". I further think that no mods should be allowed for metaverse submissions, that, or you should implement two different ladders. It's simply too big of a Pandora's box to allow in any form without requiring too much time and effort on your guys' parts to police. More importantly, you need to remove or actually balance the effects of victory type on score. As an example, if it was your intent that we play the game versus the metaverse, there is no reason for cultural victories to be scoring 1/20th or less of taking that same game and pushing it to an alliance/military victory. Even the most casual of players, if they bother to use the metaverse, are going to be disappointed with seeing their scores penalised so heavily. If balancing will take too much of your time, just remove victory condition from the formula altogether.

Like was said above, give us a more or less simple system, explain, and be done with it. Just make sure you don't leave the big gaping holes in it that we have now.
Reply #14 from Stardock Forums Top

In engineering there is an axiom:

Perfect is the enemy of good enough.

We are not saying that we never intended the metaverse to be a scoring system. What we are saying is that we believe the system is "good enough".

As time goes on, we will be continuing to enhance the game itself and as part of that we will improve the AI in ways that will have the side benefit of closing exploits. We have already been doing this and will continue to do so.

1.04 has a significantly altered way of scoring games such that difficulty level matters more than map size. And winning through technology victory will mean a lot less than winning through some other method.

Why do so many people at the top have wins via beyond humanity? Because what they do is get to a point where they are strategically secured and then basically run up the score. We have taken the view that those who we feel are doing so and submit their score that we have the right to remove individual score running entries.

But no solution would ever satisfy everyone. There are, literally, about two dozen people out of the entire GalCiv population taht we would say are really..."into" the metaverse at a level way beyond everyone else to the point where the score on the metaverse matters more than having fun within the game itself.

At the end of the day, we are talking about a single player strategy game. It is somewhat hard to feel sympathy for people who are intentionally ruining the game for themselves because they are instead trying to get a high score on the metaverse.

For those people, I recommend this game:

http://www.progressquest.com/

It's called Progress Quest.

Reply #15 from Stardock Forums Top
It sounds to me like scoring will be different in 1.04. Possibly, since the developers clearly read these forums regularly, the scoring may be changed to include a larger bonus for finishing quickly to offset the population points. That, I hope, will help the honest players better measure themselves against each other. I'm one of the people that really likes the metaverse (guess I'm a less-than-five-percenter).

I have stated before that I would like to see improvements to the metaverse, but Frogboy is quite correct when he talks about what it would take to stop people from cheating. It's extremely difficult to eliminate cheating when you make the users play the entire game on YOUR servers. Once you allow them to submit games that were once on a computer that they have superuser priviledges on, it's all over. The time consuming tasks that Frogboy stated above wouldn't stop someone who knows computing and is serious about cheating.

I don't think effort should be expended to stop cheating, but I do think that some effort to create a more robust scoring method for honest players (it sounds like this is being done) would be appreciated, if only by the small minority of which I am a part. On the other hand, the surest way to ruin a game is to try to satisfy everyone.

But only 4 out of 5 dentists recommend sugar-free gum, so if you're under 20% you're in crazy sugared gum land. -Jon Stewart

[Message Edited]
Reply #16 from Stardock Forums Top
1.04 has a significantly altered way of scoring games such that difficulty level matters more than map size. And winning through technology victory will mean a lot less than winning through some other method.

Why do so many people at the top have wins via beyond humanity? Because what they do is get to a point where they are strategically secured and then basically run up the score. We have taken the view that those who we feel are doing so and submit their score that we have the right to remove individual score running entries.
End of quote
On the other hand, this penalising of tech victories is exactly what is wrong with your current approach (and I have played extensively with the "improved" 1.04 methods). You have normalised the scoring formula for a tech victory on the parameters of a huge/gigantic map with "abusive" score running present. As such, for the casual player, it's simply pointless to go for if he care's about getting any sort of a reward. In one experiment, I spent an additional 3 hours playing the game to a tech victory without any cheese and received about half the score I did going back to a previous save and going for an alliance victory instead - twice the score for half the effort just because I didn't win with a "politically incorrect" method. That's a terrible way of fixing a scoring abuse, just flat out terrible. The penalties for cultural are even more severe and you haven't addressed this beyond saying you want to look at some save games.

What you need to do is fix what allowed the tech win to be "abusive" (pop running mainly) and stop hosing people playing for tech wins without the cheese involved.

Basically, it boils down to this: if you don't actually do something to make the metaverse a more fair, simplified scoring system, you're going to lose the majority of your score posters except that 1% vying for the top 10 slots,and that data is going to be largely useless for improving the game. I understand your view point, I just totally disagree with what you've done for the most part to address the problems as I believe they have aggravated the problems rather than smooth them out.
Reply #17 from Stardock Forums Top

Tech victories are, on a large galaxy, the easiest way to win.

Even an amateur player can win that way. Just fortify your little corner of the galaxy and outlast everyone.

Reply #18 from Stardock Forums Top
Thanks for the progressquestlink :) I'm already hooked. Can't believe i never saw it mentioned before somewhere.

Reply #19 from Stardock Forums Top
Frogboy,

Just wondering if there is any chance of a seperat metaverse scoreboard for non moders?
Reply #20 from Stardock Forums Top
If a player never runs up their score, once they hit
the highest difficulty level, and cant go higher,
any improvement in their ability would reduce thier
score because pop >>>>>> then time, which is the
opposite of how it should be.

I cant make this flaw any simplier. It has nothing
to do with cheating, it only effects the top 1% of
the metaverse players now, but it will someday
effect anyone who can beat the game and tracks their
improvement through the metaverse(regardless of caring
a whit for their overall rank), which is alot more
then just 1%.

And you can take your holier then though attitude
about your perception of my gaming intention as some
form of perversion to where the sun doesnt shine.

I have beaten the game on every map setting at
masocistic using every concievable 'method' towards
victory. Just because I only submit games that
improve my ranking does not mean that is the only
games I play. This btw is still the SECOND major
flaw in your system.
Reply #21 from Stardock Forums Top
A seperate score-board for non-modders (what I asked in my first post, thank-you-very-much) can't be that difficult, can it ?

Ofcourse real cheaters could find a way around it (mod,play,unmod-submit, sniffing, beating the CRC,...), but that would me a *real* minority...
Reply #22 from Stardock Forums Top
How do you determine if someone is a non-modder?
Reply #23 from Stardock Forums Top
I agree in all of the points you have made, Frogboy. Your statement closely resembles my humble thoughts on the matter and I support your decision. :)

Nonetheless, a question occured to me. When you say
game development is about managing limited resources. Any engineering hour devoted to making the metaverse even more robust is an engineering hour not devoted to improving the game itself.
End of quote
do you mean you will not develop the Metaverse any further at all or only very rarely? Because that wouldn't fit in with the truely amazing support you have delivered (and I mean delivered ;)) for this game so far. I hope and believe you don't mean that.

On all the other points I must say I am really happy with your decision! From the players perspective it is truely logical! :)

Good on ya! :HOT:
Fabian.

~SDC~
[Message Edited]
Reply #24 from Stardock Forums Top
Tech victories are, on a large galaxy, the easiest way to win.

Even an amateur player can win that way. Just fortify your little corner of the galaxy and outlast everyone.
End of quote

So, someone points out the flaw in your fix: that there are plenty of situations where a tech victory is actually harder to achieve than a miliatry or alliance victory, but gets half the score because you've done exactly what I said, normalised the formula on the assumption that it's the easiest way to win on a Large or bigger map, and you just don't care and come back with a non-defense?

You cannot know what the situation surrounding any given victory is so making such a blanket assumption about how it was the easiest is going to penalise just as many, if not more, players that it corrects for. I'll say it again: take ANY effects from victory condition out of your formula, they don't work, and are going to fail no matter what you may try to do.
Reply #25 from Stardock Forums Top
Progress Quest! That's really great! I love it!!! LOL
I think now I will play that simultanously to GalCiv...
Looks like GalCiv's got competition for my playtime! LOL

That's a good one!!! :HOT:

~SDC~