5 GHz WiFi is fine. Worlds had several 5 GHz WiFi networks.
The hotspot feature in Windows lacks the configurability to disable 2.4 GHz broadcasting though. And, using this feature is likely to introduce additional hassle with networking TM properly.
A much better solution would be an inexpensive but decent 802.11ac consumer WiFi router.
I am familiar with this argument. I speak from experience, though, when I say I know EPs that would really like to provide judging feedback, despite the additional effort.
Also, the whole point of making an optional feedback system as @vexvoltage suggested is to allow only EPs up for the extra work to do the extra work. If judging feedback poses an excessive burden on your event, then just don’t do it.
But even giving feedback at just a few events in a region (in line with the optional designation) would allow many teams to receive valuable feedback from an outsider perspective, even if only once in their season.
And, as the system is refined by the “pioneer” events, it will become even easier for other events to start providing feedback too.
I’d be interested in giving the paper system @Foster used in his pilot a try; could be a good starting point for expanding feedback.
Ok, I will bite here… event to event judging will always be inconsistent, even within an event different judges and lenses. I am not in favor of providing judges feedback to teams because of inconsistency.
That said, we are missing the one consistent element in a team and that is their adult coach/mentor who should be providing development feedback to the team. At the EP Summit I suggested we train the coaches/mentors better.
I agree with this, but many teams cannot rely that heavily on coaches/mentors. I know several teams that are essentially autonomous, with little reliance on coaches/mentors beyond paperwork and legalities.
Also, I really have no idea how RECF might implement any sort of widespread coach/mentor training. (Voluntary workshops can only go so far.)
As much as judging is inconsistent, more feedback from different sources is better than just one source (especially if that source may be biased and/or inexperienced).
I do expect that the new publicity around the inconsistency in judging may cause problems. However, after the initial influx in controversy, I think we will be able to refine the judging process across the board to improve consistency, and there will be less controversy overall since judging will no longer be a complete black box.
This is not new. Take three teachers in a subject and have them fill out ANY rubric and you will find between them there is a variance in scoring. This is with seasoned professionals who do this day in day out with the same students in front of them. It is a stretch to expect a group of people to ever fill out a rubric with consistency with people they just met.
The Design Rubric is only a tool for the judges to deliberate the decisions of who merits the design award based on their best opinion at the time.
I am waiting to see the new judges guide to see what new guidance is offered.
I know that. I said the publicity would be new, not the inconsistency. Many students deeply invested in VRC will have a negative initial reaction before reasoning that there will always be inconsistency. This is besides the point, though.
I think, personally, the rubrics should not be shared. They are just numbers after all. But, comments and suggestions are important to take a good notebook to an award-winning level. This would also prevent teams from complaining about awards being given to teams with lower rubric scores.
First off out of curiosity where have you seen it recommended to use hotspots/ the information you posted? @B-Kinney @anon4126930 both DWAB and RECF post very clear guides on recommendations, network diagrams, and other TM information publicly.
TM does not even need internet connection, as @holbrook clarified it just needs to be on a LAN.
Trying to take notes quickly on a phone in today’s world is just as fast for some users. I write emails all the time on a phone pretty fast and most users are just as fast on phones in today technology driven world.
I am really only interested in this thread for the potential of an electronic solution that could help ease tournaments into this solution of giving feedback. I still think all points and feedback should be given to the coaches not the students.
+100
In some instance though, a team will take feedback better from a stranger (A prophet is not without honor except in his native place, or the way a company will bring in a consultant , whichever analogy you prefer). This is why I recommend the local meet up and swap.
Good recommendations -I think if we have better training material to allow teams to better themselves, it would go a long way to improving the situation. Events are suboptimal for teaching teams about good engineering design practices.
I agree that a phone works well for many people, and eventually it will be most. Based on the judges I’ve interacted with, though, that’s not where things stand today. It is definitely moving in that direction, however.
Most of the tournaments I’ve attended already have significant technology challenges. Many times the skills scores would have been lost entirely, if not for the organizers having recorded the runs on paper. Several of the tournaments I’ve attended recently have had neither wifi nor reliable cell service. Many EPs are still struggling to get the current round of technology working in their environments, usually schools. I don’t recall any tournament using digital technology for judging. Based on observation and comments, I think it’s likely that a well-constructed paper form with a tear-off strip for judges comments (as described for the pilot feedback system) has a greater chance of wider adoption.
This is an issue in itself, is this because of poor training? Poor documentation? low funding? Not enough time? Why is this an issue?
Why what are they doing to TM? The system of TM is really robust, maybe .01% failure of being the software.
Well it would have to be WIFI not Cell service since it would have to be on a local network back to the event. Also School WIFI is an issue in itself of either only being 2.4ghz, using enterpirse WPA2 which limits certain device connections and so on. It is better to use hardwired Ethernet or home routers. Unifi makes some great low end enterprise hardware which is used at worlds and higher quality events.
Another item to print off? another item to have someone get them perforated? I am going to say this is going to curb adoption at a mass scale.
In my experience, TM for some reason just inexplicably refuses to function properly sometimes (way more than 0.01% of the time). And, in those times, TM usually just freezes up. (I don’t have bug reports because TL;DR I don’t have time to troubleshoot what seem to be weird edge cases when we need to just get TM up on another computer.)
Is electronic judging something I would like to see? Yes. I just don’t think TM is the right software for it, buggy or not.
I think something like this should just be integrated directly into RobotEvents, like the new volunteer management system. That still leaves the problem of events without internet access, though. On the other hand, this means tournaments wouldn’t have to run networking all the way to the judges’ room(s); judging could just piggy-back off of existing venue WiFi.
I mean if people are having issues and not reporting them, how are things going to ever get fixed? Can’t fix something that people are not informed of being broken. Also in my experience of TM freezing it has never been TM’s fault it has been more of a resource issue of the device (old school laptops)
Building another system to integrate into RE events creation and database to essentially come back around and communicate the same way TM does… seems a little bit of a round about way…
If this is 5ghz then why not run the TM software through it? If this is 2.4ghz shut it off.
Providing feedback is very important for students’ growth.
However, some events may not have enough experienced volunteers even to talk to every team, let alone write detailed feedback for all of them.
Similarly, having computerized feedback system could work great or could take time away from the actual judging if something doesn’t work. I’ve been to more than one event when computers/network setup would not cooperate and you have to fall back to paper and pencil.
What I found to work the best when judging, is to quickly look through the notebooks at the beginning of the day. Then, after the first round of the pit interviews, you can see which teams care about the design process and, later, take a second, more detailed, look at their notebook and talk to other judges who do the grading.
Heading back to the pits to catch remaining teams and to do follow up interviews (with the teams considered for awards) lets you talk to any other team one more time.
Judges don’t have time to talk in depth to everyone, but if I sense that a team is receptive to the feedback and could benefit from it, I will try to talk to to them.
It could be either direct feedback or it could be camouflaged as the questions in the follow up interview, and we could either talk about their notebook or actual robot design.
In some cases I would find a mentor or one of the parents and give them additional feedback on what their team is doing great or could improve for the next competition.
If I had to write it down, I would have to do a lot of of guessing, starting from if the team is even interested in it or what needs to be detailed.
Talking to people gives me much more flexibility - I could re-evaluate dynamically what team is more interested in and spend as little or as much time on every team based on their needs.
I know.
The idea was that it would be a web-based system built directly into Robot Events, not like TM.
Many, if not most, of these types of networks isolate devices from each other. And tournaments usually have absolutely no control over them, whether to disable 2.4 GHz or troubleshoot networking issues.
The point I was getting at was just to decouple a potential judging system from TM, because, away from the main scoring area, a random internet connection is usually much easier to accomplish than a LAN connection to TM (whether by school/venue networking, cellular hotspot, etc.).
A lot of events do not have internet access. This would not work for any event without internet.
This is why a lot of EP’s either work with IT departments or bring small network setups which are cheap and reliable.
School networks are normally locked down pretty tightly, limiting them to systems they know about. Their network admins are not fans of spending the day with me. I’m not a fan of needing to do domain joins to other people’s networks.
I carry a self contained network (routers, switches, access points, computers, Raspberry Pi, tablets, etc.) It’s all pre-configured, so it’s ready go. I don’t connect to the outside internet because I often don’t have good cell coverage for the hotspot. So making it part Robot Events isn’t helpful.
My network has 2 TM masters (HS/MS) and 2 TM skill computers and 11 Raspberry Pi systems. I want the event to run well, I’m not thrilled by piggy backing on TM. I also have crashes, and I do turn them into DWAB. But some of them have been self induced by trying to let other devices use my network.
As far as paper goes, we print a metric ton of paper out now to have it as backup. (Release forms, inspection forms, judges, score sheets, etc. ) Buy a fabric pattern tracing wheel to do the perforations in the sheets.
The technology problems at tournaments occur for all sorts of reasons, mostly simple logistics. Things that are easy and obvious to IT people are not easy and obvious to people outside the field. Running a network cable across a main walkway is a good example. You can tape it down with gaffer’s tape, put it under a threshold tread pad, cover that with a walkway rug-runner, and it will probably be okay. But then a student spectator rushing to get a slice of pizza can hook her foot in it when she goes around the side of the barricades you’ve placed, and the network comes down briefly, and then goes intermittent. A couple of noisy electrical connections because a 120V power cable is plugged into a failing outlet normally used by custodians running a floor machine can undo days of planning and testing. A bad USB connection on a hardwired mouse attached to a TM session can wreak havoc, causing all manner of problems, requiring Windows and TM restarts. A finicky HDMI connection on one of the Skills Field TM sessions that is incorrectly diagnosed can result in someone believing a server reboot is needed. The network can all work flawlessly when you last test it on Friday evening, and then when you come in early Saturday morning, you find all the infrastructure is powered down and offline; the IT staff which you last communicated with at 5pm on Friday is busily working an emergency upgrade to fix a vulnerability they were told about at midnight. I’ve seen all those happen, and many more.
Many tournaments are held in school facilities. School systems don’t usually fund weekend IT on-site event support. So you can work with them ahead of the event, but on the weekend you’re on your own. The network, IT, and cybersecurity people (which might be “person”) can’t be there, because it would cost too much to do it. You can ask ahead of time, but that won’t make it happen. Those people have plans and lives too, and we just don’t fund them to be there.
I mentioned the lack of cell service because, even assuming the hard-wired network can reach the internet and update Robot Events, your attendees still can’t use VEX VIA to track their schedule. You need more monitors so they can see the standings, and you need to produce paper schedules after the teams check in and get through inspection.
That’s the sort of issues I’ve seen at tournaments, and that’s why I think paper is a pretty good idea.
I don’t think the concerns regarding judge feedback in this thread are a problem that can simply be solved with technology. It seems to me it’s primarily a concern about creating the feedback in the first place by the judges. The delivery method (electronic or paper) is incidental. If all the other issues are worked out and the only remaining issue is a requirement for an electronic system to distribute feedback, then I’m sure that can be worked out by RECF. I guess my point is there’s not much reason to continue bikeshedding on the nuances of TM and wifi networks etc. as it relates to judging.
As a side note, I will once again repeat my plea: if you have issues with TM at an event, please at least let us know (using our support email address preferably). It doesn’t have to be root-caused or debugged by you, just a note to us “hey we saw TM do this and it didn’t seem right” along with the log files if at all possible. Even if we can’t do anything specific with the information you provide, we can at least observe if there’s patterns developing to see if there’s issues in a certain spot. For whatever it’s worth, I don’t think we received more than a single-digit number of reports over the entire 2018/19 season of any significant issues despite there being 1000s of events. I’m not saying that to claim nothing is ever wrong, but rather if bad things are happening, no one is letting us know.
Anyway, back to your regularly-scheduled debate on awards judging and feedback…