|
|
First, I think the obvious critique is that the average is supposed to be based around 1500. 1500 is a fairly insignificant number, especially considering the range. There are a number of options that I think would be better than this depending on a handful of factors.
Second, and my main critique is that, PSR(Player Skill Rating) does a poor job of actually measuring player skill. Not only that, but, since the "K-factor" is calculated based on PSR, PSR gets exponentially worse at measuring skill as time goes on(games are played). I will explain:
I find it is important to give definitions for terms in an argument, since differences in definitions is often the cause of contention in arguments. --So, Player Skill(a.k.a. skill) is defined as a players ability to increase the chance that their team will win. i.e. if you swap out player A for player B and the chance the team will win goes down, then player A is more skilled at the game than player B. Also note the instance based definition of skill, where by player B can skill be more skillful than A if B has a good game, where A would have had a bad game. e.g. B played the game skillfully. That is the definition of skill I'll be using, so if you don't care for that definition, you might as well stop reading now.
I'll start by comparing PSR to %wins for estimating skill. Since %win shows a players tendency to win when added to random teams, it does at least a reasonably good job of showing skill, considering skill is the difference in win likelihood when added to a random team. A difference exists between those two statements, but there is definitely a strong logical and statistical correlation between them.
PSR works something like %wins, in that the value goes up with wins and down with losses. PSR, however attempts to be more accurate than %wins, by acknowledging and attending to the fact that, when you are added to random teams, not all games are created equal, since skills on a team can vary which will have an impact on how influential you were in the outcome. e.g. you can be 5-0 and suck because your teams always pulled you up, also, you could be 0-5 and rock because your teams drug you down.
PSR attempts to deal with this by essentially weighting the outcomes of each game based on the skill of the people in the game. The problem is, that skill evaluation is the PSR value its self. PSR gets calculated using PSR.
When error is introduced into these two systems, %wins %error remains constant, while PSR's %error is expounded by using its self in its own calculation.
Example: assume we have 10 players who have all played 100 games, not specifically with each other. Assume 1 of those players had 1 completely erroneous game, and that there is no other errors currently in the system. I will say that a game, in which the player in question was the MVP of the entire game(both teams) and was on the losing team, is a 100% erroneous game. This would both optimally confuse %wins and PSR.
Assume that those 10 players play a game together, and the game added no additional error into the system. %error for %wins decreases overall since the same amount of error is averaged over a larger game set. i.e. (100*9+99)/100 < (101*9+100)/101. However, since the player with 1% of error accounts for 1/10th of the calculation for each persons PSR change, 0.1% of error gets added to each persons PSR change. thus, even though this new game had no error in it, the PSR scores of this game added extra error into the system anyways. i.e. after that game 9 players have a very slight amount of error in their PSR(meanwhile 0%error in their %win), and 1 player has roughly 1% of error in their PSR(and likewise 1%error in their %win).
This merely showed how 1 flawed game out of 1000 impacted the next un-flawed game. Now imagine a system with a constant amount of error. e.g. for every 10 games played there is 1 games worth of error (a.k.a. 10%error). %error in %wins will remain at a constant 10%error, while %error in PSR will grow exponentially, so even if you assume %error for PSR starts below 10% because the PSR numbers start-out accurate, it will eventually grow larger than %wins %error. That also assumes PSR numbers start-out accurate, which they clearly don't since everyone starts at an arbitrary 1500.
Third, 1500 is in fact NOT average. The only way 1500 could remain average is if the total PSR remained constant, changing only with the creation and deletion of accounts. I believe there is both a mathematical proof and historical events out there that prove total PSR is not constant for me. Since the total PSR in the system can grow, and PSR is supposed to represent a unit of skill, when the total PSR grows it would suggest the HoN community as a whole got more skilled, which makes a whole lot of sense, BUT it means that the PSR value that represents average is constantly changing with the fluctuations in total PSR.
Having a system in which the average skill level is a constant number has advantages over a system that doesn't when viewing it from a singular person's perspective. If I have 1632 PSR, am I good or not? I don't know unless someone tells me what average is, and no one has, so PSR is rather pointless other than comparing to other people you meet on a micro scale.
Also note that, since S2 proclaims 1500 to be average and starts new accounts at 1500 even though it's not, if PSR has grown over time, which it has, then accounts created now are started at a lower skill value than accounts created in the past. This again makes sense on a global aggregate scale, but punishes new individuals on a micro scale.
Forth, Ludo-logically speaking, a scale that has essentially a 500 point radius, and starts at 1500, isn't very positively re-enforcing. It implies that you can't even come close to doubling your skill level, which is not encouraging, which in turn is against a major tenet of good game design.
TL;DR version: "PSR does a poor job of actually measuring player skill." Or I think so, read only half of it.
There's 3-4 things wrong with the current implementation. I'm still editing my post on the PSR system's flaws, but I intend to point these out explicitly.
Your post isn't going to be convincing to anyone, though.
1600 is what the K-factor's middle is based around, while 1500 is the starting PSR. There are some minor artifacts(this combined with the K-factor clamping lets you 1v1 2 1500 accounts to get both above 1500; but there's only a small range of PSRs where you can do this trick), but it's not actually a serious issue.Third, 1500 is in fact NOT average. The only way 1500 could remain average is if the total PSR remained constant, changing only with the creation and deletion of accounts. I believe there is both a mathematical proof and historical events out there that prove total PSR is not constant for me. Since the total PSR in the system can grow, and PSR is supposed to represent a unit of skill, when the total PSR grows it would suggest the HoN community as a whole got more skilled, which makes a whole lot of sense, BUT it means that the PSR value that represents average is constantly changing with the fluctuations in total PSR.
You're assuming that PSR's "skill level" is linear. A difference in 100 PSR points will turn a 1v1 into roughly 80%-20%; "player skill" is in some sense exponential in the difference is PSR.Forth, Ludo-logically speaking, a scale that has essentially a 500 point radius, and starts at 1500, isn't very positively re-enforcing. It implies that you can't even come close to doubling your skill level, which is not encouraging, which in turn is against a major tenet of good game design.
To put it another way, the hypothetical top-of-the-world chess grandmaster with 3000 ELO ranking is not "twice as skilled" as a 1500 practicing club player; his win % would be so vastly higher that you'd have to get him drunk or high to beat him.
If it were possible for your PSR's error(difference from true PSR) to grow exponentially, then in particular your PSR would grow exponentially. The K-factor is clamped between 10 and 40 so this isn't possible.while %error in PSR will grow exponentially, so even if you assume %error for PSR starts below 10% because the PSR numbers start-out accurate, it will eventually grow larger than %wins %error
Let's be honest, HoN's K-values are pretty stupid.
The difference of 10-40 is twice as much as 16-32, and it caps at relatively low levels (1440, 1680).
I have returned
HoN should really take games played, em games played, total kills total deaths and total assists into account. a person with 1 kill and 1 death have 1 kdr, which is the same as a person with 5000 kills and 5000 deaths...
All of these statistics are taken into your account; you can see them by right-clicking a person's name and clicking "view stats."Originally Posted by kaboozeroll
It's Public skill rating.
1600 is an equally arbitrary and inaccurate number, so maybe I'm missing your point here.
I am not assuming PSR is linear, I am assuming peoples perception of PSR is that of a linear progression, which is the rightful assumption. Assuming anything else from a calculated user stat would be illogical. Positive re-enforcement is based off of perception, which is what I was talking about.
%error isn't strictly equal to 'difference from true' value, and certainly isn't specifically positive. If this is confusing to you, then consider how significant digits are calculated through equations. I believe this will lend a relative parallel. Lose of information isn't a delta, but an uncertainty of truth. If you multiply an inaccurate number by an accurate number, the inaccuracy of the first number gets carried over into the result, regardless of how accurate the second number was. The 'true' value isn't even known so the %error isn't a difference. My "true" PSR and my given PSR could be the same number even if the %error is very high. The chance that the two numbers will be the same decreases as the %error goes up. That's really what we are talking about, not how wrong your PSR is, but how little you can trust that your PSR is right.
Last edited by CobaltBlue; 11-14-2010 at 12:53 AM.
You are saying that PSR isn't exact? How does that make it a bad system? If you can come up with a way to measure exact skill you will be a billionaire. It's an estimate that should move around in a bell curve centered at your "actual skill". Randomness evens itself out it doesn't compound exponentially.
You say it isn't linear but people "rightfully" assume it's linear. How does one rightfully assume something that is wrong? PSR is a standard model and most people would assume as such.
% wins is meaningless. When you are saying the error in % wins error from what? What is the baseline % wins that the error is from?
PSR is essentially unitless. The units are standard deviations not some "unit of skill".
Last edited by AtheistGod; 11-14-2010 at 03:29 AM.
I find it is important to give definitions for terms in an argument, since differences in definitions is often the cause of contention in argumentsPSR = Public Skill RatingPSR(Player Skill Rating)
Maybe i am stupid but i do not understand whats the point out of this post
You talking about error % , but all that tend to balance as if there is an error in your PSR (ex you s*ck but reach hight PSR carryed by good team) . Your PSR just gonna tend to drop in the next games as your skill is less hight than what your PSR represent.
Because everything tend to balance the error in the system slowly goes down and that is what make PSR a good scale
PSR can be reset and is , therefore, unreliable at mid ranges.
My experience so far: You need to play a semi/ hard carry to win a game in low tiers. This sucks hard, most of the time I played support. Just started to play carries to get as higher KD and PSR. But the other problem is: You end sometimes with four hard carries in a game :/
So what to do? Prevent people from picking what they want? :/ Kinda bad...
Playing BD would be an option, but in low tier no one wants to play BD :/
SM isnt working, tried it twice and both times people left before game startet ( The countdown was showing they got a DC, I hope)
My best bet would be: Close custom matchmaking for one or two days and rework SM maybe this helps :/
--------------------------
Oh ****, my signature just vanished with Scout :/
--------------------------
The only thing I don't like about this PSR system is that if I play with a friend with lower PSR and we lose. He or she will lose more psr than I will, eventually increasing the gap between us.
I've even seen cases where people can't get up from their current PSR-level because when they win they win ~2 psr and when they lose they lose ~30.
This means he or she must win 15 games just to catch up with one loss, and that's quite stupid to be honest. Sure, he or she must be bad in order to get that low, but to get up (after doing some practise and with more knowledge and skill) he or she must use the costly feature of reseting stats. Seeing there'll be tons of effort to raise oneself from these levels.
So what I mean is that low psr people shouldn't lose -40 when they are up against higher tier players. It should almost be the other way around to be honest. A high tier player playing with a bunch of low-tier players will not lose any PSR. Better give this stomper the disadvantage of risking a -40 psr loss because he can't grow the balls to play against players with the same PSR.
Which brings us back to the "playing with friends" situation; Sure, you shouldn't be punished for playing with your friends, but neither should the low PSR people that CAN'T get up be punished because the gain is so low at low level games.
The lowest I've been at is 1350 or something, I don't know might be more. All I know is that at 1700 rating I usualy gain/lose 5 psr a game, while at the "bottom" I gain or lose 20. This is a big flaw in my opinion.
Solution to this?
Well a suggestion would be to make the whole team lose/gain the same ammount of PSR from a game. Based on the win% chance.
This will not be too accurate but better than the current if you ask me.
(Do tell me if I am missing something here please)
Example: Team A has 80% win chance and overall higher PSR. The gain from a win in this situation should be like 3 psr, the loss from a loss in this situation should be like 17. If Team B wins they gain 17 and if they lose they lose 3.
What now? Am I contradicting myself in saying that the low tier players shouldn't be punished if a high tier player is in their team? Well they will lose equally much from this loss. Meaning the gap won't increase.
ĖDIT: Regarding how this system measures skill I don't care, because there will never be a system that can tell perfectly how skilled a player is by a simple mathematical function based on stats from games. The stats does not handle good positioning, said player's accuracy with spells and abilities and so on.
Last edited by GregerMoek; 11-15-2010 at 06:17 AM.
Stats are like women in bikinis, they are interesting to look at however they don't show you everything.
How about PSR +/- for each game being calculated considerint the TSR (true skill rating). I fond TSR the best way to know the skill lvl of a player. My friend is 1750 and has only .8kdr so people think hes not that good. How about that 2.6 Wards per game, or 11 assist a game,etc.
Also, with TSR, i can stomp pub very easily. Im 1670 or something and I often play with my friend whos 1750. So we need 1650+ in the other team to balance. Sometime i get 1600+ with TSR below 4. What does that tell me, they play with good friend but they personnaly suck ballz. They can have a good game once in a while with some STRG Heros but if they got a WH, WS or anything they will be completly useless. For example, i've got a friend who 1600 but is totally noob with most of the hero(but is good with pred, plague). The reason is that he is playing with some of his other friends who are really good and they pub stomp. I often tell him, go play by yourself some game so youll loose cuz when i play with you you totally disbalance the game.
I prefer playing with a 1400+ with 4.5 TSR than a 1700 with 4.0 TSR.
So one way to counter jackass like me who balance the game PSR wise but unblance TSR wise would be to integrate TSR to the game. What do you think?
Anyway you know what I mean. I agree that calculating PSR with PSR is a problem, so what about integrating the TSR into the game since its really good to calculated +/- psr for a game???
PS mother tongue french so sorry for bad english
The TSR arguement is stupid, I don't care about kills and assists, they don't win me the game, a win wins me the game. Thus a measure of skill based on wins should be the most accurate. That said, I do think TSR should be shown in game, but I'm not sure how it should be integrated into the AB.
I don't understand all the whining about PSR. In my experience, PSR is an accurate indicator of how likely an individual is to win me the game based on prior wins against enemies of their skill level.
And if you make it so the PSR gain/loss is not symmetrical then you have PSR constantly decreasing, which is not desirable.
You are not clever and unique for knowing how to play. Everyone can play Devourer as well or better then you. Just because you instapick Devourer and get firstblood doesn't make you good.
psr is crap because players are crap, when terrible idiotic morons stop playing this game we can accurately create skill levels for different players... Theres a difference between being a moron and simply being a lesser experienced player.
PSR is 100% accurate (if this was a 1on1 game).
Since it isn't, teams will often win vs pubs even though their individual skill levels are higher. Or a bad player will win with his 4 better friends. Or an A1 team captain picking the right heroes/strategy will win the game for his team by default. And if you want to use any other stats than raw wins, those can always be manipulated... see TSR and wards placed where so many people who'd have had 0.1 wards placed per game will now have 10 wards per game for their rest of their life due to ward of revelation abuse/spam at the end of games.
So... just take it as it is, PSR being a rough indicator of one's skill, that might not be worth much sometimes, but works most of the time rather fine.
_ _ _ _ _ [Video] Pharaoh - Rocket (Tormented Soul)
_ _ _ _ _ _ Anubis _ Force Commander __ Levi _ _ Broodmother
_ _ _ _ _ _ _ _ Dream Heroes | Forum Heroes | Items
_ _ _ _ _ [Video] Rampage+Tablet vs Chronofield
TSR tells as much about someones skills as PSR. Play support heroes -> bad k/d ratio -> bad TSR -> you get a codex scout telling how bad your TSR is.
TBH the PSR system isn't that bad. The fact that most people win 50% of games means it's fairly accurate...
The point of the diminish effect for overrated people is so they can't gain PSR by playing people 200 PSR below them. Your suggestion makes the situation you describe worse not better.
What you describe as a "flaw" is the system being lenient towards lower PSR people. If they get better they can rise much faster because of it. If it was equalized it would take a 1200 player who became as skilled as a 1500 player about 4 times as many games before their PSR is accurate. At high PSR it assumes changes in skill will be less rapid and so stability in PSR and less effect of RNG is better than being able to reach your real PSR faster.
Posting Permissions
