From: "Jeff Bihl" Subject: On Ranking Violation Percentage Date: Fri, 5 Jan 2001 19:18:46 -0500 The problem with picking one criteria to judge all retrodictive systems by is that not all retrodictive systems are written to meet the same criteria. In my opinion the purpose of a retrodictive type system is to be sure that an individuals criteria are applied exactly the same to all teams. The only way that ranking violation percentage is a fair criteria by which to judge two systems is if two individuals used minimizing that percentage as their criteria for their system. The main reason why I do not believe in minimizing ranking violation percentage as a criteria in my own systems is that is that when you rank teams in this manner a game will either matter a lot (move the team that won ahead of the team it defeated) or it won't matter at all and be totally ignored. The biggest problem with this in the long run is that it does not punish really bad losses enough nor does it reward really big wins enough. A strange fact is that a really big win (beating a team ranked much higher than you) has a greater chance being ignored than small one (beating a team just above you). I will demonstrate this later. The other problem I have is than you stand absolutely nothing to gain by playing a team ranked below you and absolutely nothing to lose by playing a team ranked ahead of you. In my opinion if there is something to lose there should be something to gain and vise versa. I try to keep risk and reward balanced. Here is a demonstration to illustrate my point. Lets make five teams named A, B, C, D, and E. Those five teams have played round robin and split with each other all going 2-2. Lets set up another group of five teams which will be teams F, G, H, I, and J who have played round robin and split with each other all going 2-2. Now let's say that all the teams in the first group (A-E) went 5-0 in their games in a full round robin against the other group (F-J). As Mark has noted, a team's record can be used effectively as a retrodictive rating in pro sports because the schedules are usually balanced. In this particular ten team group winning percentage works perfectly as a ranking because all the schedules are perfectly balanced. They all played each other one time. So a system whose goal was to minimize the number of upsets in the standings would have the same results as the conference standings: 1. (tie) A, B, C, D, E 7-2 6. (tie) F, G, H, I, J 2-7 Each group is still essentially tied in a system whose criteria is to minimize ranking violation percentage because you can rank the teams in each group of 5 in any order and you will get the same number of violations. There is no way to separate them by this method alone. I will say they are tied by this method. Now lets add an 11th team, team K, to the mix. Team K beats teams A, B, and C in its first 3 games. Now team K is number 1 in the system whose goal is to minimize ranking violation percentage. Teams A through E are still essentially tied for #2. There is still no way to separate the 5 (A-E) by this method alone. A, B, and C would not be punished for the loss at all. Teams F-J are still tied for last. Let's consider to what happens to the rankings of a system whose goal is to minimize ranking violation percentage in a few scenarios for team K's remaining games. Example 1: Team K loses to team D in its fourth game. Team D is #1, Team K is #2, Teams A, B, C, and E will be ranked according to who beat who in the 2-2 round robin split, and F through J are still tied for last. As you see Team D was helped by a decent win and team K was hurt by a decent loss. Example 2: Team K loses to team F in its fourth game. Team K is not hurt by this loss and team F is not helped. K is still #1, A though E #2, and F through J tied for last. There is still no way to separate F-J by this manner. So we see that team K is hurt by a decent loss but is not hurt at all by a bad loss. Team D was helped by a good win in example 1 but team F could not be helped at all by a great win. Example 3: Team K loses to teams F and G in its fourth and fifth games. Just like the fourth game against F, the fifth game against G doesn't change the rankings at all. Example 4: Team K loses to teams F, G, and H in its fourth, fifth and sixth games. This is the strangest situation of all. Team K has lost three games to above average teams and lost three games to teams that are exactly as far below average as the teams that they beat are above average. So they have an even record against an exactly average schedule. One would think that they would have an average ranking. That would mean being ranked 6th among the 11 teams (behind A through E and ahead of F through J). In the system whose goal is to minimize the number of past upsets average is the one ranking that they CAN'T have. There is equal reason to put them either first or last, which makes 3 of their games in violation either way. If they are in the middle then all 6 games are in violation. If you take an average of best possible rankings for minimizing the ranking violation percentage they do end up average but the average of the possible results violates the criteria the system started with in the greatest manner possible. Example 5: After a loss to I in the seventh game after the games in Example 4 K would drop all the way to last. The wins in the first three games mean nothing. This is just my opinion and why I choose not to do it this way. My system has a parameter that is essentially standard deviation of a Normal Distribution. The higher I raise the number the more like the system above it acts. I like to keep it at a value where it keeps the risk/reward for playing hard or easy games in the best balance. That's just the way I think is best. I'm sure others would not. I don't put much weight in judging a system by looking at the results. I think you look at the algorithm and see if you agree with the way teams are rewarded and punished. I believe that if you agree with that then you should accept the results.