While another thread had talked about how people plan on doing their digital books, not much was said about how they plan on validating the dates as the rubric requires. While Google docs and OneNote do have timestamps, how do the judges plan on validating the dates. While edit history like Google docs can be accessed, they are usually not exact to every day and when teams update their books every couple days, these edits will be condensed into one.
How do you guys plan on having the dates be validated in your digital notebooks?
I don’t think they mean validate entries in the literal sense that you could actually prove when you wrote the entries. I think its more like “date your entries when they were written please”. I mean, they were fine with just written dates in a physical notebook that could be whatever the signer wanted them to be, so I don’t see why it would be different here. Besides, it isn’t really possible to validate a timestamp in most cases, and judges aren’t going to want to spend their time checking dates (assuming they even could)
Let me preface this by saying I am a coach, Event Partner and sometimes judge. I have been a part of several RECF meetings this summer about the changes in events for this season. It has been mentioned several times that digital notebooks will need to be timestamped, and the updated rubric says that timestamps should be “verified”, but in all the meetings that I have been in, there has never been any mention about how we, as judges are going to actually do that. RECF is often intentionally vague in the way it words tings like judge’s guides, to give event partners the ability to host events with the resources on hand. These resources include judges. The judge advisor for an event you are at may be a teacher from the school who has been trained and is good at organizing things, but knows very little about robotics or engineering. On the other hand they could be the CEO or an engineering project manager for a local engineering company. These two individuals will likely view this requirement very differently based on their levels of experience, and you never know which judge will be looking at your notebook. The new judge’s guide does say that when in doubt, judges should err on the side of the teams (believe the timestamp), but I can anticipate a situation where the judges are trying to make final decisions for the Design award, and they have it narrowed down to two notebooks- one where the students just typed in a date, the other used software that provided actual, real timestamps. I think it’s easy to figure out who wins the tie.
As a coach, I will be encouraging my teams to use software that provides an actual timestamp for entries. I try to encourage them to use as many tools that are used by real engineers as possible. We already use CAD software, and put coding projects on GitHub, to me this is just one more way that we can model working like engineers do.
Also, it is important that we are having these discussions now, because making a change mid-season will be very difficult. What if you realize after your first competition (which let’s face it, will probably be very late this year) that the judges don’t consider the dates you typed on entries verifiable, and you need to move them to another source that is more robust. Now you have months worth of “non-verified” dates from before you made the move. Yes, judges should give you the benefit of the doubt, but the notebook that has true verifiable dates back to the beginning of the season will likely beat you every time. In a very competitive region, this could make a big difference in your season’s awards.
I don’t think that you can make this assumption. Remember that in a bound engineering notebook the pages are numbered, so there is at the very least a forced chronological order. Now I know that you can skip a few pages for something you know you need to do, but haven’t had time for yet, and you can back date those entries (not that you should, but it happens) but for the most part, especially after the first event, the notebook is probably chronological.
In a typed document you can just scroll up and insert things that never really happened, basically fabricating things, and writing a date on it means nothing because you can edit all the dates after it. In my opinion just dating entries does not count as a timestamp.
100% agree that for this reason alone this needs to be decided ASAP and not left as a surprise for students at tournaments. While having things “vague” for the sake of flexibility makes sense in some areas, I think this is an exception as many students put 100s of hours into these notebooks. In my opinion, students need to have confidence that their approach will be honored at all tournaments throughout the season (local, regional as well as Worlds) and if there are levels of timestamping that will be evaluated (ala emerging, proficient, expert) that needs to be stated up front so students can take that into account when planning their notebook process.
I also believe the comment in the addendum that one can provide a PDF will lead students to rationally assume that using software where one can type dates and generate PDFs (i.e. Google Docs, MS Word, Text Files) is acceptable. Again, if that is not what is meant, I think more clarity is needed so teams aren’t penalized for doing what is stated but perhaps not intended. Just my .02
The ones I have seen other teams mention are commonly Google docs, and Microsoft OneNote, and the lesser used elabFTW. Google docs and OneNote allow you to write whatever date you want in the pages, but they both have an edit history that can’t ever be changed. You can save these as a PDF and just give screenshots of the edit history I guess.
I’ve never personally used elabFTW so I’m not sure how verifiable those dates are.
I’m still not sure whether judges will accept any of these at all, we need some clarification from the RECF and acceptable softwares.
Google docs also has a tendency to compress some of those verifications and it would be very difficult for the judges to use the edit history to do any real verification. A day of edits could be spread all over the document.
I have the same question regarding revision history in these programs… What would students print out for timestamps when they have to turn in bound notebooks? From fiddling around it seems like those revision history screens don’t lend themselves to printing alongside the documents they point to.
I’m glad teams are having the discussions and figuring a good process now, and I agree with much of your post - especially the focus on having students learn and use real-world engineering and collaboration tools.
But I think the sentiment you concluded with on the decision paragraph quoted above goes 100% against the Judges Appendix and RECF’s specific statement that students should be given the benefit of the doubt.
Given the 2nd team made reasonable efforts to provide some sort of proof/documentation for their notebook version history, if the judges made the decision solely on which software was used for the Digital Notebook, both the EP and Judges would be going against the RECF Judges Appendix very specific instructions. That means both teams get the full credit related to their timestamp entries.
Surely, we can expect our EPs and Judges to make a determination of award winners based on the overall Engineering Notebook Rubric and interview, and not penalizing a team that did not use or have access to tool that provided auditable digital timestamps.
Our teams are considering Named Versions in Google Docs (to avoid the compression issue), then include pictures or screen shots of that history as appendix (or possible as pages right before a competition) in the cases where printed copy of the document is required. Some of our teams have been working on the design process/notebook for quite some time, others are just getting started.
Edit: there would also be a URL reference to the public accessible google doc just in case the Judges had any doubt about the authenticity of the provided proof of revision history/stamps.
2nd Edit: The teams exploring this identified a Named Version limit of 40 versions. So we have a little more work to do on the compression issue / process.
Well adding on to that, we really only need to know if the 5 points are given or not depending on what software you use, or if you could simply print out your entire notebook etc.
Agreed - that’s what it comes down to. I believe RECF is very clear that any tool can be used for Digital Engineering Notebook (quotes below pulled from the Judges Addendum).
Digital Engineering Notebooks may take any form that is accessible by clicking a link without any special software other than a web browser or PDF reader.
The 5 point bonus applies regardless of the type of Engineering Notebooks (I realize the debate is “confirmed time stamps”, but the student benefit of the doubt I think precludes the need to have any auditable digital timestamps.
The 5-point bonus is applied to all Engineering Notebooks with “confirmed time stamps” regardless of the type of Engineering Notebook. The Judge Advisor will use reasonable discretion to determine whether a time stamp is chronological and will apply that standard consistently to all submissions at the event. When there is doubt or question regarding a time stamp, the benefit of doubt should be given to the team.
You can definitely print out your Digital Engineering Notebook for the event when the event has in-person judging and requires a physical copy of the notebook.
Where a team has a Digital Engineering Notebook and can attend an event with standard event judging, the team must print out their notebook prior to the event and submit a physical copy.
Well, VAIC team has decided to use GitLab. It provides real, verifiable time stamps , and the kids on that team are very experienced with all sorts of technology, so are comfortable with it. They looked into elabFTW but felt that it “felt too buggy” (their words) and decided they liked GitLab better.
However my VRC teams are pretty inexperienced this year and are mostly younger kids who are new to robotics. We still haven’t made up our minds. We use MS Teams to collaborate remotely, so OneNote would be the simplest choice, as you can easily integrate a notebook into a Team. It does timestamp when entries are made, but these timestamps can be changed, so are not verifiable on the level of elabFTW or GitLab. They also discussed using a Google Doc or Evernote, but for us OneNote would definitely be easiest to implement. I wish there was a better actual software solution that was user-friendly enough for those kids, but as of yet I haven’t found one.
I am considering having the team members write up a contract to have as the first page in their notebooks that says they agree not to alter the OneNote timestamps, that they are true and valid and have them sign it and use it as the first page in their notebooks. Also, they will be asked to export their entries weekly as PDFs and those will have a true timestamp. We plan to use freeTSA to do that. I guess I should also mention that I teach this as a class, so the weekly PDFs will be how they earn their grade. I can understand how it might sound like I was a crazy, super controlling coach otherwise,
The weekly PDFs are a great idea for providing further proof of the benefit the doubt case - regardless of the platform, I’ll mention GitLab to our VAIC team since they are just getting started.
Well yes you can change the timestamps that are visible on the page, but you can always open the actual page history to see what day edits were made on it. The actual page history cannot be changed.
What if you just took a picture of each page and upload it to a google drive folder as you complete the pages?
Google Drive would have the date on there and that would be easy for the judges to spot check a few pages as they wanted to. You could put a QR code on the front of the notebook that links to the google drive folder.
Your digital notebook could then just be a pdf made from those images.