Does the current scoring and ranking system encourage mediocrity?

VEX V5 Technical Support UNOFFICIAL Tech Support
1

We are finding that very good robots are finishing low in the qualifications ladder. This is the situation.

1/ robots get the score of the loosing allaince as their Schedule Points SP (this is used as a tie breaker).
2/ robots with the same number of wins are ranked upon their SPs in the qualification rounds

The above is used to try to resolve the ‘easy game/hard game’ issues that occur when the randomised qualification matches are created. This seems to be a fair way of resolving the issue.

The problem arises when you simply have a very good robot that clean sweeps most of the time and an average robot that just wins every time. With the current scoring system the less able robot will be 1st and get first choice of alliance partner. Teams are being forced to hold back to keep their SPs up and those that do not hold back fall down the qualification ladder.

Obviously holding back is an exciting strategy in the qualification rounds but should we not be trying to get the teams to go ‘all out’

Would reversing the system work? Possibly but then robots that get an unlucky draw would suffer. At least we would then be encouraging robots to clean sweep and not hold back.

Please could we have your suggestions that we could put to VEX?

2

Well, if a robot completely sweeps their side of the field clean (leaving the other alliance with no points and them with no SPs), then they’re not playing the game very well and I’d say that they’re not a very good team. The robot might function well but if a team is doing this they obviously have not spent much time thinking about game strategy.

In my opinion, the fact that the rankings work the way they do is an indicator that the objective of Clean Sweep is not to clear out your side of the field (despite the name).

Basically, I don’t think the rankings are flawed - they’re part of the game. Anyone who isn’t considering them when playing isn’t playing very well. If the rankings were changed to reward teams who completely clear out their side of the field, then that would be an entirely different game. Changing in the middle of the season would be unfair to those teams who have factored the rankings into their overall strategy and robot design.

3

The current system does not encourage mediocrity.

It does reward drivers/operators who think carefully enough to attempt to avoid winning 100 to 0, if they can instead win 75 to 50.

The drivers/operators who keep the scheduling points in mind; and make increasing them a part of a comprehensive humans+machines strategy; are typically not the mediocre folks.

There is no perfect scoring metric. I do like the compromises imbedded in the current one.

In the current metric, both defeating a tough opponent with a score of 51 to 50, and blowing out a tough opponent with a score of 100 to 50, earn a team more SP than blowing out a weak opponent by a score of 100 to 0.

Also, using a comprehensive strategy, and going easy on a weak opponent by winning 100 to 50 instead of 100 to 0, is rewarded (and the losing team gets to leave the field with some fun and inspiration under their belts).

Sounds OK to me.

Blake

4

Maybe I’m too competitive, but I think that holding back is both risky and not worth it. I know it’s nice to let the other team not get blown away completely, but thats competition. True competitiors blow away and learn by their losses, not being satisfied by someone holding back. By holding back it’s risky for the teams since they both want to win by a safe distance rather than being close and potentially losing their lead.

I would rather see people win by being good competitiors, earning every point they get, and going to Championships due to hard work on driving and design then by the “you were nice to other teams” factor. FIRST has the phrase Gracious Professionalism, and its a great thing to follow - compete hard and win, but don’t be a jerk about it. It never says to hold back, and competition is all about giving your all.

My suggestion is to reverse Max Score and Strength of Schedule
in the ratings so high scorers are put higher, and then sorted from there instead of being potentially penalized.

And I would like to add that being nice and all is not a mid-season shock and strategy. Changing the rules to reward hard work for competitions is not changing the game completely - it’s just people need to work to win and not rely on some safety cushion when scoring. Strategy helps you win more, not less. I don’t know about everyone else, but I’ve built and practiced to win, not to hold back and risk losing.

I would love to hear why IFI came up with this idea so we can all clear this up, because right now it seems I’m thinking differently then others.

5

The problem is you’re trying assert your own definition of what “the best robot” is, but that definition doesn’t match what IFI has defined it as (implicitly through the rankings rules). IFI’s rankings basically say that the best robot is the one that beats teams who are also very good scorers. In fact, I don’t think it says anywhere in the description of the rankings that the reason is “to be nice”. If the rankings were changed to reward teams who score big (without regard to opponent score), mediocre teams with a weak schedule would end up ranked higher than excellent teams who play many really good teams and win each match. (That is, after all, why SPs are “Strength of Schedule Points” - an attempt to rank teams by how hard of a schedule they played).

Let’s look at a scenario. Team A is awesome, regarded by every person in attendance to be the best team at the event. Teams B and C are very good and roughly equivalent. Team D isn’t so great and really needs some work.

Match 1: Team A versus Team B. It’s a close match, but A wins 99-80.

Match 2: Team C versus Team D. Team D has issues, and doesn’t move the whole match. Team C wins 100-0.

If we took your suggestion and ranked by max score first, team C ends being ranked #1, while team A ranks #2, even though everyone at the event believes team A is the best there. Is beating a robot that doesn’t move a “better” win than beating a really good robot in a close, hard-fought match? I don’t think so personally. If this rankings system were used and this scenario occurred, then there would be a bunch of posts on here complaining about that.

Bottom line is that no rankings system is perfect, and there will always be scenarios where the rankings don’t really put the correct teams at the top. However, in my experience, the current rankings system gets pretty close as long as there are enough qualification matches played by each team. If there aren’t enough qualification matches played, then there isn’t enough data and any rankings system will produce poor results.

6

I’ve never agreed with SPs/RPs, but until that changes, teams just have to deal with it and come up a tournament strategy.

Some things we’ve learned:[LIST]
*]Looking at a tournament schedule, it is pretty easy to identify which matches will be harder and easier. Play SPs in the easy ones and go all out in the harder ones.
*]At the end of the qualification rounds, taking risks might boost your SP ranking a few spots.
[/LIST]

We’ve gone n-1 (n wins and one loss) in three tournaments where we scored points for the other team in the lost match. Especially in the bigger ones…that hurts (your ego + your ranking). 254A’s current strategy is to go moderately all-out. It doesn’t always guarantee a 1st seed, but it usually guarantees a spot where we have some leverage over which alliance we end up in.

7

I think it’s also important to remember that going 6-0 is always a good idea, even if your SP is not perfect. I think only four of 260+ robots at Elevation Worlds last year went 6-0 at qualifying. Even if your SP was lower, you would still have finished in the top three in either division with a 6-0.

I did see some really dominant robots in earlier regional competition sandbag and help their opponents score last year, but I can’t say that I saw it at Worlds.

(By the way, I agree with Dave and Gblake’s comments in this thread, so I won’t repeat them.)

8

I got from the previous posts that “backing off” and “being nice” was a viable strategy, and I was replying more to that than anything. I wasn’t trying to redefine “best,” I was just trying to say that if the rule enforces backing off then its absurd. There should be no penalty against those who work hard to score well. But I guess if it’s not going to in the end and will only boost those who won in tough matches versus easy winners, then it’s worth it. I just didn’t want any unfairness for those who have high-scoring matches just cause they’re good.

9

This was discussed numerous times in the last year in our robotics club, and we came to the following consensus:

The intent is threefold:

  1. Make it fun for everyone. Sure, there are a bunch of excellent robots out there, but there are also the robots that are not as good. The students who built these robots put much of their time and effort into these robots. If the rules were changed to make it so that one team could just crush another, then it would be no fun for the losing teams, and they would probably not return the next year.

  2. Add a new strategy element. Keep in mind that the SP/RP only matters for the qualifications. Thus its effect is twofold: does a team that has the capability to win want to make all the games close to be a top-ranked team and an alliance captain, or crush and hope that the alliance captains notice them and pick them? It is a tradeoff which teams must think about. In my opinion, it makes the qualifications that much more exciting, as it adds that extra thing to think about, and makes the game just a little more complex.

  3. Truly rank teams. This is much like what Dave said. There were about 150 teams in each pool at worlds this year, but each team only played with 18 others (6 allies and 12 opponents) in the qualifications. Many of the qualifications are heavly one-sided. In the case where there are two good teams playing each other, the SP/RP system shows this.

With all this being said, I don’t believe it truly had that big of an impact in worlds. The teams who wanted to be alliance captains balanced their scores to do so, and almost all the teams picked by the captains were the teams who blew out all the other teams (we did this for our picks).

This system has some issues, but I believe they can be fixed. I also believe it is the best way to make it balanced and fair for all teams, not just favoring the best right away.

10

Folks,

I’m a little grumpy at the moment, so factor that in.

Also, this can be a polarizing topic. If you wade into this pool you need to bring nearly bullet-proof agruments to the debate.

This isn’t a football game. It isn’t a poker game. It is VRC. Maximizing qual match QP and SP is part of how you win this competition’s on-the-field portion.

I agree that it makes sense to suggest redefining VRC; but it makes no sense (to me) to fuss about holding back, or to call a correct strategy absurd.

I assert that the current QP+SP metric makes the #1 Qual Match ranking harder to get and hold than all other simple, feasible metrics I have seen discussed.

You have to look through the correct end of the telescope to properly discuss winning the field-matches portion of VRC. Knowing which end of the telescope to look through is a sign of excellence, not mediocrity. That it might be hard to rack up a lot of SPs is a good thing. Good teams will accumulate QPs. Excellent teams will accumulate QPs and SPs.

Ensuring that you get big SPs out of every qualifying match is not holding back, and it certainly isn’t absurd. Instead, it is playing right up to the edge.

If their allies don’t include SPs in their match strategies, the teams I mentor get annoyed and wish for better allies.

I enjoy seeing teams that are good enough, get right next to that edge. The best ones get really close without going over. The mediocre ones either are afraid to get close to it, or mistakenly go over it.

If you have “built to win”, that is probably biggest part of getting ready for the individual field matches. Congratulations!

Now, expand into “thinking to win the whole competition” during those matches; and also start working on the rest of your on-and-off-the-field strategy for earning the Excellence award.

Be one of the best!

Blake

11

Alrighty, sounds good. I wasn’t intending to come across as naive or single focused. Don’t get me wrong, I love extra challenge and strategy. I was just trying to say that it is risky to stay near and I don’t necessarily want some great teams to lose the chance of winning because they’re good. Not to mention I did kind of just brief over that portion of the rules despite reading them several times, so I guess I have and my teams have quite a bit of thinking to do. I guess the rule also has to do a little with confidence - if you’re confident to win, you don’t have to totally bash teams, you just need to play with that cushion some to go futher than just a flatout winner, but rather an all-around competitor.

12

Krummel’s team sounds like a team “my” guys would want to have for an ally. See you on the field, and in the pits.

Blake

13

This is the fantastic thing about VEX. It makes you think deeper than just the surface rules indicate.:rolleyes:

I can see more teams playing right up to qblake’s ‘edge’ after reading this topic!:slight_smile: Now the scouts will really have to work even harder to recognise the difference between a team that is really good and ‘edging it’ and one that is working hard to stay there.:smiley: he he the mischief that will cause!

So now we need instantanious mathematicians as coaches! Wow big words there!

14

For Elevation matches, it was not that hard to get a rough approximation of the score (when there was more than about a 10 point differential), but took a lot of effort to try to get the exact score for SP. For Clean Sweep, it seems it will be a lot easier, as each ball has a fixed value, so you can make a differential each time a ball switches sides, leaves play, or is locked up.

15

This topic is all but pretty much settled, but I’ll add my $.02.

I think that the wins/losses system is pretty decent. No arguments there.

Within there, the other points or whatever they’re called, add a little “fun” to the game.

Think about it this way. If you’re playing a close match, where both sides are somewhat even, play your hardest and don’t look back. But, if after the first minute, it’s overly obvious that this match will kind of be a blow out, just back off a bit. It’s not “limiting” yourself or even playing “down” it’s playing strategically.

Back in Lunacy, I saw this one match where shooters with turrents were dominating match, and the score was like 80 to 10 with roughly 45 seconds left. Then the turrenting robots turned around and shot into their own trailers to bump the other team’s score. Not “playing down” it’s playing smart. :smiley:

And no need for instantaneous mathematicians, if you see like all but 6 balls on the other side of field, you can kinda guess that you’re winning. :stuck_out_tongue:

16

My opinion on the ranking points is mixed

They’re a pretty good tiebreaker but:

In cleansweep, it doesn’t separate the really good robots from really bad ones very cleanly.

Two teams of very strong robots will score very close to how two teams of weak robots will.

The difference in ranking points doesn’t seem to be all that much in my opinion. In general, for a match, the difference between scores is 20-30 or so. As long as all the balls are knocked to one side or another, most teams will get close to the same amount of ranking points in the end.

As for scoring, my suggestion is to find where the majority of points in a match come from and find a way to quickly tally them. For example, in elevation, 48 points, which is pretty much 60-80% of the total points scored during a match depending on the level of the robots are determined by ownership of the goals. Calculator coaches aren’t necessary.

17

I love it ‘Calculator Coaches’. Can I use that.:slight_smile:
How about an overhead vision system that keeps a realtime track of the scores?

18

i like it. lets start production say… tomorrow?

19

Cue the spooky music :wink:

It just dawned on me that Team 960a (Krummel) and Team 42a (“one of my teams”) did become allies. :eek:

In the Halloween Northwest MD Regional, third-ranked 960a drafted fourth-ranked 42a, and 12a.

The 3-team alliance earned the tournament’s Tournament Champions title.

I sure wish I had known to buy a lottery ticket back on September 28th when I wrote the quoted statement above. My crystal ball was exceptionally clear that day.

Blake
PS: I hereby declare that this outcome is irrefutable proof that the current scoring and ranking system does not encourage mediocrity. :wink: :rolleyes:

20

Blake,
Your crystal ball may be good but check your memory chips. No teams were undefeated. Five teams were 5-1-0.

Jon