SP ranking system ideas

VEX Robotics Competition Discussion VRC
1

Hi

I thought it would be interesting to get other ideas on how the SP ranking system could work.

I guess one reason the current system is used, is to help prevent one-sided demolition games occurring in qualification. If this is a reason, then I think it is a reasonable one, otherwise new/weaker teams could become very demotivated.

However, it seems that the current SP system rewards teams who face weaker opponents and/or have very strong team mates through the tournament. In other words, a “lucky” draw. Not only can teams bank easy WPs but can also bank a lot of easy SPs. If you have a strong team mate and weak opponents in a match, you can control and boost your SP for this match, especially if the opponents cannot/do not carry out much de-scoring. Changing this to kill points still rewards a lucky draw.

My idea is to reward teams that have a hard draw - the SPs would be be based on the WPs for the 3 other teams involved in each match. You would add together the WPs of your 2 opponents and subtract the WPs of your team mate.

For example, your opponents have 2 WP and 4 WP (not so strong) but your team mate has 10 WP (is strong). Your SP would be 2 + 4 - 10 = -4 SP. This means, if you have an easy game, you are rewarded with WP, but penalised with SP. Your SPs would not be fixed based on the current WPs when the match was played, but would keep updating as further games are completed and each teams WPs are updated.

This is something that would need to be built into tournament manager to work. I think it would be great, as teams could glean a lot more from the rankings after qualification. Teams that had been carried, or just had a terrible draw, would be a lot more obvious.

Other ideas please, Paul

2 Likes
2

Interesting… and not a bad idea.

To be most useful the SP’s in this algorithm would need to be updated as the tournament wore on. Consider that for your first match everyone has 0 wins and 0 losses so we don’t know who is strong yet. That can only be determined at the end of the event… so a team’s SP would be moving up and down throughout the event depending on how their former partners did. The dynamic wouldn’t necessarily be bad, but it would definitely be different.

I’ve been involved in competitions using the current system for a decade now, and don’t really have a problem with it. I like that it discourages blow-outs… it really makes the point that its not cool to run up the score… but that doesn’t mean that good behaviour cannot be encouraged in other ways.

So I see no need to swtich, but if I had to switch, I could see this model being a decent one.

Jason

3

Similar thoughts here throughout the year. I agree with dtengineering that your system makes more sense, but the true SPs can only be calculated by the end of the tournament. In addition to that, you have absolutely no control over your SPs and can be easily hurt in the rankings if you have an easy set of matches.

I think the current system works okay, but instead of the opponent’s score (an absolute value), it could be the inverse of the difference between the two scores. While this wouldn’t fix the issue where teams can still score for the opponent to increase their SPs, it would at least make those matches even with the truly difficult matches that are more in the 70-60 range of score.

4

It doesn’t really discourage blowouts. Losing by 200 points may be demotivating, but so is seeing your opponents start to score points for you.

And I disagree that the SP system isn’t broken. It may have worked moderately well in previous seasons, but it doesn’t work well at all for Sack Attack. Even in Gateway and Round up I think the assumption that higher scoring games meant better quality robots only worked at low-level events where most robots weren’t capable of doing very much. When most teams are good, I think SPs reward an easy schedule which is the opposite of their intended purpose.

5

I thought I would give a more in-depth explanation of why my mentor and I, as well as quite a few others I talked to, think the SP system needs a look at for this years game.

In case people don’t know, the purpose in SP is to give some kind of reward in terms who have a difficult qualification schedule, as it seems logical that versing stronger opponents will result in a higher score for the losing alliance, as both alliances will score lots.

HOWEVER: A common strategy for teams to use is “farming” SP. When playing against weaker robots, a team may score points for the other team in order to gain a higher number of SP, potentially resulting in a higher ranking. This has obviously been present in previous years competitions, but wasn’t really a problem, as a difficult game usually still resulted in more SP than a farmed game. Also, farming for SP was just another skill competitive teams should have.

I believe however, that in Sack Attack the SP system is actually fundamentally flawed. Due to the nature of descoring in this game, competitive matches will tend to involve robots which can descore very quickly. If descoring takes place from both alliances, it follows that the SP given to both alliances will be very low. However, if versing an opponent with a very weak descore, or no descore whatsoever, a team is more free to safely farm SP’s. I know a lot of teams at the Asia Pacific Championship (including myself) in qualifications were using the strategy “Don’t descore anything unless the game is close”.

Basically what I am trying to say, is that in my experience so far with this years game, an EASIER game results in more SP than a harder game. Considering SP are meant to measure the strength of the opposition teams you have faced, I believe this current SP system is very flawed. I know that personally at Asia Pacific, I found myself getting very few SP in difficult qualification matches, and getting much more SP in matches where the opponent could not descore.

I know it’s not very “routine” for Vex to change a game part way through the season (there is kind of precedence with programming skills before worlds last year), but I believe this is a serious flaw in the game and will result in a less enjoyable experience for teams at worlds, and I seriously do believe a change is necessary. I know some people will agree with me and probably a lot will disagree, but I know there were a lot of teams at Asia Pacific who were feeling the same way.

6

Yes, the SPs can only be calculated (accurately) at the end of the tournament. However, scouts can start to see the trends as the tournament goes on, and alliance selection does not happen until after the final matches have been played.

Surely a game where you do have control over your SPs is flawed if the concept of SPs is designed to let them compensate for the draw you find yourself in?

If you have an easy set of matches, then you should be able to get plenty of WPs.

7

For tournaments that have multiple undefeated teams, this new system can really hurt teams that deserve the first seed but got an extremely easy set of matches and could end in 3rd-4th seed. I have no alternative to this though, as obviously in a perfect world the randomness of match difficulty wouldn’t be a factor. Maybe we should just make Worlds a 2 week event and have all teams play 30+ qualifying matches :smiley: this is why I like the league system so much.

8

I know I have said this a few times but I personally think that the best tie breaker instead of Sp would be the total wins of all of your opponents.
This would cause beating better teams to rank someone higher than beating worse team.

9

I’d be curiious to see how much difference it would make… seeing as how match results for major tournaments are posted, it wouldn’t be that hard for someone who was passionate about this (I’m not that passionate about it, or I’d volunteer) to run some models and show what it would have looked like, either using the originally proposed system, or the “easier to explain to a visitor” system of tallying up the sum of your opponents’ victories.

I have to say, however, I did see a wonderful example of learning come about from the “point farming” strategy where an experienced team (from a very experienced school) didn’t quite do all of their math, and rolled back to park on their square with a five point lead and a few seconds left in a match.

Then their opponents dropped all their pre-load sacks on the unoccupied coloured tile and tied the match with one second left on the clock.

Ooops. That’s how we learn the importance of math.

Jason

10

Here’s an idea that I find intuitive but that no one has suggested yet:

Make it illegal to intentionally score for the opposing alliance.

If the primary purpose of the SP’s are to give teams with a difficult schedual an advantage in ranking then this will allow the current system to function as it was intended.

As some have mentioned a secondary purpose of the SP system is that it (kind of) prevents blow out matches. On this I would argue several points:

-While the current system mathmatically prevents blowout matches, it is still blatently obvious to the audience that one alliance is destroying the other. This is pretty much impossible to curcumvent no matter what rules are put into place.

-I think that having robots score for the opposing alliance is confusing for the audience and counter to making the game easier to understand from a spectator point of view.

-From a competitor point of view I think that being grossly outscored by the opponents is much prefered compaired to having the opponents score for you. Having your opponents score for you is kind of the ultimate slap to the face and while currently a part of the game really presents a much greater opportunity for the spread negative feelings then just winning by a lot does.

My 2 cents

Regards, Bryan

11

I bet I could incorporate the idea of assigning a match Kill Point score with my power rankings, which will be implemented at my next competitions :slight_smile:

12

While i do see issues with the way SPs are counted I don’t know if changing the system would be a good idea

Most people seem to think that something that looks at the difference between the two scores would reward close games better. While this is true it wont stop teams “farming” by scoring for opponents in easy games because they will still need close games, maybe they will just stop scoring so that there opponents can catch up to reduce the gap and increase the SPs.

If you did it the other way and tried to reward teams who get a “unlucky” draw by giving more SPs to teams that get smashed. You would remove all the insensitive for a team to score if they are paired with a square bot rookie team while playing a game against robots that are contending for tournament champions, they they know they are going to lose so they want to maximize the gap to increase there SPs

But I have bigger issues then what system we change to. I was helping run Asia Pacific over the last few days and had many teams come up to me and asking all sorts of questions. One thing I didn’t get asked about was SPs because everyone competing understood them. If we changed the SP system I would imagine that a large number (>40% maybe) would fail to understand the new system and would arrive at worlds with no idea of how they worked.

Another questions VEX would have to look at before changing the SP rule for Sack Attack would be what rule would work for the next game, and the game after? It would be very confusing if key rules like how SPs are counted changed every year.

The current rule may have some issues, but its easy to understand and I think that is one of the most important parts of any VEX rule.

13

We threw some ideas around in the car on the way home from the tournament on Friday after we realised that SPs were working opposite to the intention for SA. The final conclusion that we came to is that no system is going to be ideal, given that large number of different goals/intentions that VEX/RECF try to meet. Those goals/intentions are similar to those for having 3rd alliance partners, and given that this is the system that was chosen several years ago, it is much better to stick with this system than to further confuse teams and replace it with a similarly non-ideal system.

I think a better solution is to restrict division sizes, and try to get more matches played. For example, with 400 teams at worlds, play with 8 simultaneous divisions of 50, so that each team gets more matches and the qualifying matches produce more accurate seeding. While this would be more tiring and difficult for teams, it would certainly show who the best of the best are.

14

At our tournament yesterday, we had a hard schedule during qualifiers. We went up against a handful of robots that were very good so the strategy ended up being just win and don’t focus on strength points. We ended up winning all of our matches and even “farming” SP’s when we had easy matches. After 6 matches we had only 279 SP’s and we were ranked 2nd, behind a team which had a much easier schedule and was able to pull over 350 SP’s.

I do think that the current SP system is backwards and I’m quite positive that a lot of the more competitive teams will benefit from a new system. I am in no way complaining, right now, it is what it is, but for in the future, hopefully during Sack Attack, I would expect to see a slight change in the way SP’s are calculated and gained.

I agree with smaller divisions, like you stated above, but I do not think Vex will go that way for world’s.

15

Right now my mentor, my brother and I are working on a new SP system that we are calling TRSP (Truly Representative Schedule Points). My brother is actually creating a python script which will be able to pull the results of a competition from a robotevents page and calculate each teams TRSP value.

At the moment TRSP:
-Is only effected by how difficult your schedule is
-Cannot be farmed

Sound good?

I would appreciate it if you could PM me a link to the tournament you are talking about’s robotevents page, and also, if you could give any of examples you can remember of teams that you think had difficult, easy, or average qualification schedules. This would just allow us to have an extra case study to see how well our system works, although preliminary testing on the results of Asia Pacific 2012 is looking very promising.

I do understand that it is extremely unlikely that the current SP system will be changed, and the complexity and volatility of the TRSP system we have created is certainly another barrier. I do however, think our system will be a great aid in scouting. You can know which teams have had easy qualifications, and which have had tough qualifications.

1 Like
16

I’m looking forward to seeing this new “TRSP” system! Do you have an estimated time of completion? I am in no way rushing you haha I’m just wondering.

17

Sadly my brother is working most of the day. There is a possibility it could be finished tonight (our time, ie. within 10 hours or so), but there is also a chance it could take a couple of days, depending on the difficulties we encounter. Thanks for being patient :).

18

It seems like no matter what kind of system someone can think of, there will be good teams that end up with a disadvantageous schedule and will suffer because of it.
As it is now, a team with an easy schedule is rewarded with the ability to farm SP points. Teams with difficult schedules suffer in that they likely can’t score as many SP points due to descoring.
However, with a system that rewards difficult schedules, the problem hasn’t been fixed. There will still be good teams who randomly end up with an easy schedule and are left out. Its the nature of the competition and the purpose of the Excellence award.

19

I don’t see it that way. SP are called schedule points. They are supposed to show how difficult a schedule was. If SP are not showing how difficult a schedule was, then they are not filling their desired function.

SP are supposed to reward teams that have a difficult schedule. Isn’t it logical that a team who wins every match with a hard schedule should rank higher than a team that wins every match with an easy schedule?

Maybe what you point out shows why SP shouldn’t be the second form of ranking after WP, because they are so based on luck, no matter on whether the SP system works or not. However, I still think the SP system does need to be changed, otherwise we are rewarding teams who have an easy schedule.

20

I personally feel that the best way to handle this is in such a way that it wouldn’t be a major rule change halfway through the season. The best way to do that would be to make it illegal to purposely score for the other team. It is pretty clear if a team has accidentally dropped some in the wrong trough, versus they are deliberately scoring for the other team
That being said, I do think we need a rule change for next year. I think that for this the system proposed by Torqueative and Telemascope would be the best solution. Granted it may need to be tweaked but it’s a good start.
Lastly, to all the VEX Robotics officials I know are reading this, could we have some sort of official response? Thanks!:slight_smile: