The Q&A Needs to be Fixed

sazrocks · February 6, 2020, 12:04am

The Q&A Usage Guidelines state:

Questions will (mostly) be answered in the order they were received.

We know that your question is important to you, and you want the answer as soon as possible. Most questions submitted by Wednesday evening will be answered on or before Friday of the same week. Most questions that are submitted on Thursday or Friday will be answered on or before Friday of the following week.

However, some questions may take additional time to review internally, and responses may take longer during high-volume times of year. In order to ensure the consistent quality of all answers, there are no guarantees that questions will be answered before a given date. Therefore, if you need an answer before your competition, we would advise posting it 1-2 weeks in advance, not the night before, to ensure a higher likelihood of a response.

This is a reasonable policy. However, it has not been adhered to in the slightest this year. Very few Q&As within memory have been answered the same week or the following week. A minority have been answered within 2 weeks. And there are a disturbingly large number of Q&As on the High School forum that have remained unanswered far beyond this time. Here are some examples at the time of writing:
https://www.robotevents.com/VRC/2019-2020/QA/443
https://www.robotevents.com/VRC/2019-2020/QA/445
https://www.robotevents.com/VRC/2019-2020/QA/409
https://www.robotevents.com/VRC/2019-2020/QA/449
https://www.robotevents.com/VRC/2019-2020/QA/466
https://www.robotevents.com/VRC/2019-2020/QA/467
https://www.robotevents.com/VRC/2019-2020/QA/468

All of these questions are at least one month old, far in excess of 1-2 weeks and yet have received no response at time of writing.

Q&A#292 was asked over 8 months ago. At time of writing, that puts the question as being asked in June of 2019. I might be able to accept that this Q&A had been simply missed except for two factors:

There are a number of other VEXU Q&As that have also been left unanswered for an unreasonable length of time.
In the High school forum, Q&A#443 brought attention to these questions, gave an explanation for why they might not have been seen, and requested that they be answered. Q&A#443, along with the VEXU Q&As, is still unanswered as of writing despite being asked almost 9 weeks ago.

How is the Q&A supposed to function when questions are left unsanswered for multiple months, if they are even answered at all? This tendency, compounded with the confusing and inconsistent rules and rulings this year leads to an immense amount of confusion and inconsistency in rules enforcement across different regions, which is detrimental to VRC and VEXU as a whole.

It didn’t even used to be this way. Those members of this forum who will remember that before the Q&A moved to Robotevents, questions were answered relatively promptly, and in order. It was fairly rare to see a question stay unanswered beyond 2-3 weeks, and virtually none were left for over a month, let alone the 8+ months that we have seen this year. Is it an organizational change or the Robotevents system that has led to this drastic degradation of the Q&A? Continuing along this line, there are a number of other ways in which the Robotevent Q&A system is inferior to the old forum system:

There is no way to determine exact dates that questions were asked on. Instead the dates are relative and are labeled in the increasingly vague day, week, month, year system.
There is no way to preview a question before it is posted. This leads to formatting often being completely broken when initially posted, and requiring that a post be edited repeatedly until the formatting is fixed
There is no way to know if or when a Question or Answer has been edited. I know for sure that certain Q&A answers have been edited from their original content, yet there is no indication of this and I am sure that there are many other answers that have been altered that I am unaware of.
There is no way to sort questions by date answered. This leads to uncertainty as to which questions I have seen the answer to, especially when questions are answered far out of order as they have been lately.
VEXU Q&As inexplicably do not show up on the VRC Q&A forum when they are asked from the VEXU page
Formatting tools are fairly limited, especially when considered to any modern forum.

These issues may have been excusable last year when the system was new and still being developed, but we are nearing the end of its second season of use and as far as I can tell no improvements have been made.

Will someone at the GDC please explain why these regressions have occurred? I honestly hate to be so negative about this, but I and many others have waited for quite a long time now and no improvement has been visible. Please GDC, fix the Q&A for the sake of the competition and those who are invested in it.

Sylvie · February 6, 2020, 12:07am

YES!!! +10000000 yes. Too many QA questions have gone unanswered for too long. This needs to be remedied. The GDC do have jobs, yes?

vexvoltage · February 6, 2020, 12:32am

Just another note the Ep Q&A, is also not being answered in any official capacity, or if it is its months old.

lacsap · February 6, 2020, 12:33am

I thought my answers were ok…

gbr · February 6, 2020, 12:36am

They’re decent answers, but the ones from GDC are gold.

lacsap · February 6, 2020, 12:44am

GDC does not answer EP Q&A - so guess I am all you got

[ok there are a lot of EP helping out on the EP Q&A, which is how it was structured to begin with - to allow peer help… VERY different from Official Q&A for VRC (middle and high school).]

vexvoltage · February 6, 2020, 3:37am

Pascal your answers are amazing, it would be nice to see official answers because we might be doing something some way even though it officially is done another. Even though you are 99% correct there is times we’re an official answer would be great or at least mark your answers as the solution if nothing else, don’t just neglect it.

JoelLynfield · February 6, 2020, 7:59am

Have we found another reason for DROW to be on the GDC. On a serous note this is something that I would expect to be done someone should take responsibility in the GDC.

Grant_Cox · February 8, 2020, 7:10pm

Hi all, Chairman of the GDC here.

I’d like to be upfront about one thing - I take full responsibility and accountability for the delays this season. Many people on the GDC contribute to Q&A answers, but ultimately I am the one who clicks “post” when an answer is ready. There is no excuse for the way this season has gone, and as a former competitor myself I would probably be making the same post if I were in your shoes. By our own metric that I wrote in the Guidelines (quoted at the beginning of this thread and bolded for emphasis), I have failed you. So, please accept this apology, in the form of an attempted explanation for the delays and a peek behind the curtain at what goes into Q&A responses.

1 - “We’re busy”. This is the first reaction I typically have when someone comments to me about Q&A delays. It is one of the most common cop-outs in human nature, and it is easy to get caught using it as an excuse. Any time you have not done something, that means you were doing something else.

The GDC is not the full-time job for any of its members, including myself. It is probably my favorite part of my job, no doubt. However, an unfortunate reality of life is that how much you “like” something does not always correlate to how much of your 24 hours it receives in a day. The typical GDC member is, by definition, a valuable subject-matter-expert of the VEX or RECF workforce in some capacity - which means that they/we may be uniquely suited, and therefore occasionally re-prioritized, to other tasks that support our mission. When that happens to too many of us at once… delays happen.

2 - A staple of the VEX Q&A has always been its intent as a general helpful reference for rules implications/clarifications. If we gave one-sentence answers to every question, it would help a lot with point 1. However, I don’t think that would benefit anyone.

The part of answering Q&A’s that takes the longest is not actually the root question. In most cases, if you ask a GDC member “hey, what happens if Cubes cross the line in auton”, we can probably answer in 30 seconds or less, “I think that means you lose auton”. (I do this every year at VEX Worlds, during the Open Forum and Driver’s Meeting).

Rather, it’s the fact that that answer by itself is pretty much useless as a long-term reference. Head Referees and teams need a background that you can use to make your own judgments. This step - pulling together combinations of different rules, debating over the verbiage (to make sure we don’t mis-use the word “should”), and trying to answer future in-the-moment questions - takes substantial time and mental energy. It is not the easy way out, but I feel very strongly that it preferable to one-sentence “read the manual” answers.

3 - Y’all ask… a lot of questions, so they need to be organized and discussed before answering. Some people have asked if the RobotEvents system is the cause of the delays. Ironically, the RE system was developed specifically to improve the efficiency on our side (and therefore speed/quality) of answers. There are new backend tools where we get notifications for new questions, discuss in-line, vote on proposed answers, etc.

With that being said, this thread has definitely brought to light a major issue with VEX U. When we were originally working with the RE developers to create the Q&A system, there was a lot of debate about how to handle VEX U. Most VRC answers apply to VEX U, but not all; some VEX U answers apply to both VRC and VEX U; some VEX U answers may look like they apply to VRC, but would actually not. We took a stab at combining them, but it is now clear that the way in which VEX U questions are handled by our back-end is not working as intended. (I am going to be completely honest - I do not recall seeing sazrock’s linked question that was posted in June). There is no excuse for this, but I assure you that I am going to personally look into this for next season.

There is nothing I can do to take back any event difficulties that occurred this season which would have been mitigated by a then-unanswered Q&A; for that, I sincerely apologize. The GDC cares deeply about the integrity of all VEX competitions, and I will be reviewing our internal processes ASAP so that we can return to expected response timelines before the end of this season, and maintain them moving forward. (and, I will be writing some answers this weekend!)

sazrocks · February 9, 2020, 11:03pm

First, I want to thank you for taking time out of your weekend to answer this post. I know you are pretty busy; it means something to me that you personally answered. I do still have a few questions though.

I personally already knew this, but I think there is a non-insignificant percentage of the community that does not. Perhaps a bio page on the GDC and its members would be helpful? Personally the only person that I know is on the GDC is yourself and that the GDC as a whole is made up of both RECF and VEX employees. I think a little transparency here would help to bring a face to the GDC and remind the community that you are all still just people who have a long list of responsibilites.

This is a good point and helps me to understand why the current delays have happened. However, this seems to imply (and correct me if I’m wrong) that the entire or a majority of the GDC is consulted for every question posted, which seems inefficient to me. Even with the move to RobotEvents reducing the number of redundant and low-quality questions, there are still a number of questions that could easily be answered by a single member of the GDC. As an outsider, this appeared to be the practice when Karthik was the one answering questions here on the forum. Many questions were answered quickly and simply, seemingly by Karthik himself, while the more complex questions took a bit longer and had more formally written answers that seemed like the entire GDC had taken part in.

(emphasis mine)

This honestly seems a bit confusing for me. 2 months ago now (December 6th IIRC), I posted Q&A#443, which specifically brought attention to Q&A#292 and other VEXU questions; indeed, my question has Q&A#292 as its first link. I understand not getting a notification for it or the other VEXU questions as they were posted in the VEXU forum, but surely you would have gotten a notification for Q&A#443, and when you read Q&A#443 you would have seen and clicked on the link to Q&A#292? I honestly don’t see how the GDC would not have seen Q&A#292 unless literally no one even read Q&A#443 until now, which would be very frustrating to me as its author.

This is exciting to hear, and I look forward to the Q&A returning to schedule.

Once again, thank you for your time, and I look forward to the many Q&A answers that will be out in the coming weeks.

TeamTX · February 9, 2020, 11:58pm

@Grant_Cox
I, for one, am certainly glad that you have addressed the issue and plan to move things forward, and I appreciate your behind-the-scenes look into what goes into answering the official Q&A. I would like to make a few additional comments, in an attempt to give some insight to the GDC about the issues from the other side.

Please, please, PLEASE consider just answering exactly like that (without the “I think”, of course). That is so unambiguous, so succinct, and so VERY helpful - had that been an answer to the first question showing that the issue was unclear, I genuinely feel like it would have saved us all from months and months of back-and-forth discussions, incorrect ref rulings, and multiple Q&A questions trying to clarify.

I fully understand that you are trying to provide some deeper, more long-term adaptable answer that has the added value of the full mental focus of the GDC behind it, but I think that you are VASTLY underestimating the usefulness of a simple, straight-forward answer. After all of the official Q&A responses regarding this particular area, we were still discussing it as recently as a week ago in great detail trying to work though the nuances of what the GDC was trying to say.
https://www.vexforum.com/t/incorrect-refereeing-of-sg2-via-sg7/74745/7

Consider perhaps posting official follow-up specific questions with deeply thought upon, fully fleshed out answers that can be more useful for long-term reference (and don’t have to follow along with whatever the exact original question might be).

True! And some of them probably shouldn’t have been asked “officially”… Lol. Too bad there’s not a way to have a first line of defense where some of those types can be weeded through and quickly answered less officially.

That said, I have noticed that sometimes when the GDC chooses to answer by giving a non-answer (something like “look at this other one and re-submit if needed”), they don’t acknowledge that often the new question has attempted to pull out some very specific nuance that the originally answered question did not. When the reader sees this, it often looks like the GDC didn’t notice that the question was different. After reading your detailed information on how the answers are discussed, I assume that someone during those discussion has called out the difference and the GDC has determined it doesn’t change the answer. It would be helpful if they clearly stated that in their response (even if it is just to restate the difference and indicate it doesn’t change the answer).

As a final note - I want to stress again what @sazrocks mentioned regarding the dates. It would be immensely useful to have original posting dates on both the questions and the answers (and perhaps adjusted dates if either of those are modified). Even if the GDC just puts the answer date in as the first part of their response (although having some way to sort by answer date would be awesome). It is exceedingly difficult to make sure that you see all the new official responses out there; EPs and refs with busy schedules are unlikely to be up-to-date on all the potential answers that pop up on long-past questions asked.

Thank you again for your time, and for the hard work of the GDC!

kmmohn · February 10, 2020, 12:35am

Here’s an idea: perhaps create a panel of trusted mentor/coach/volunteers who are zealous “manual geeks” that can screen all the questions (we know there are at least a couple of us (@holbrook ) who keep an eye on the forums for RTM questions. A panel of volunteers could screen questions, flagging them for three categories: RTM, Easy-to-answer-but-need-something-official, and Tough-questions. The screening panel could write the RTM answers, sending them to the GDC to quickly screen, approve (or move to a different catagory) and post the RTM answer. This would reduce the number of hard questions that will need deliberation by the GDC, and potentially speed things along so the GDC can prioritize.

OscarMNOVA12 · February 10, 2020, 12:49am

I opt for community voting on the Q&As to rank by priority and date

technik3k · February 10, 2020, 4:49am

Where are the olden days when Karthik would answer a couple dozens of questions in the morning and the forums would buzz with excitement for the rest of the day?

We understand that the new system must be much more convenient for Refs and EPs, who may need to quickly search through the official answers and save or print them as a PDF with one click.

Also, among the stated reasons for restricting asking permissions to team’s registered contacts and later extended to EPs and refs, was to reduce repeats, improve question quality, and

I find it that by having an isolated Q&A system, that not everyone is aware of and not everyone is going to periodically check, actually increases repeated questions asked here on the forum. It is great for sparking conversation as stated in the goals, but the new system doesn’t make it easier for people to follow the latest developments in the official Q&A, thus less awareness and more confusion, and repeated questions.

If it is too hard to integrate Q&A search into the vexforum search interface, could there be, at least, quick links added to the forum hamburger menu that lead to the current game manuals and Q&A portals for both VRC and IQ games?

lacsap · February 10, 2020, 10:53am

Great feature idea! Another one I would like to see is a link from each rule in the Game Manual to a search in the Q&A for answered questions about that rule (tagged).

alex531 · February 19, 2020, 6:58am

Does this mean that all the VEX U Q&A will not be answered this season? I get it that most Q&As are not relevant anymore since newer, similar questioned Q&A are posted and answered. However, teams may still want to repost their questions in high school Q&A knowing that their original question will never be answered.

Side notes: QA 431 is still not answered to this date, however it was referenced in QA 467

This is an extension of QA431 www.robotevents.com/VEXU/2019-2020/QA/431 and QA432 www.robotevents.com/VEXU/2019-2020/QA/432

In general, we would point to the answers provided in the two Q&A’s you have linked as our overarching direction to…

Does this mean that those QA are potentially answered but just never published?