If RAM and CPU are the problem, maybe a different DB or model is needed? Or some indexes?
I keep coming back to Apache Couch DB mainly because I want to get my hands on this. You would have to index a few things and I am not sure of the update process for a new fetch of the data from robotevents. But you could do some deeper analysis.
That’s one of the easy things about JSON is that it’s supported in most languages I’ve already got Javscript and PHP covered, I know of at least 3 apps under development using it (so that covers Objective C and Java :D).
It’s not with the database, that’s perfectly fine and efficient and not an issue. The issue is with creating the JSON output since that requires making the whole array first and then converting to JSON and that’s not really feasible as the number of entries of data are continuing to rise exponentially (currently there are 50,000 matches - I guess probably about 100,000 by the end of next season). Also by keeping the total RAM per request down it means more concurrent requests, less chance of PHP reaching its limit, etc.
I haven’t really ever looked into CouchDB, I might do but I’m not too sure if it would provide a benefit in this case?
Any possibility of implementing the update times like LegoMindstormsmaniac requested? I’m pulling some of the data to a local database for OPR/CCWM calculation and it would be nice to be able to just pick up data since my last update instead of having to pull all of it again.
I can’t seem to get the nodata flag to work with get_teams. I was able to confirm that the nodata flag works on all of other queries (Matches, events, awards, etc.). When I go to api.vex.us.nallen.me/get_teams?nodata=true, I get:
Sorry about the lack of reply before now. I’ve been trying to think of a way to realistically do this but I’m not really sure what the best way to implement this would be? (Completely open to suggestions about this!)
Sorry about that! I found a few bugs with get_teams earlier today and fixed them but it seems I broke it when searching for all teams, fixed now!
Also recently I properly added VEXU - even though the API had support for this the data wasn’t actually being pulled so that’s all go now!
My system only uses parts of it now for skills with php, javascript, and R, but the more I look thought the api, the more I realize I should use in the future to make my life WAY easier.
One small thing, and I don’t know if you have this but for the events api, could you add when the data for that event last changed? I think this might be part of what others were hinting at before, and could be extremely helpful.
I just added OPR / DPR / CCWM values to the API, these can be obtained through “get_rankings”.
I only updated this for all events within the last hour, so it is possible some things might be wrong (I checked my results with both Vex Via and Overclocked’s site and they matched up though). But let me know if you find anything odd!
Hopefully this makes things a bit easier for people who wanted this information
This has been great for scouting and everything, but I have no idea what OPR/DPR/CCWM and TSRP’s are. Maybe add these definitions to the “The Data” tab? Thanks.
I noticed on the following page that there is a paramtere repeated with different descriptions. I am working on a VB interface and needed to know what these were supposed to be or how they are differentiated.
This is incorrect. Using “field=1” will include matches scheduled on fields with the name “1”, likewise for any other number. @Robo_Eng_13, the second “field” should actually say “scored.” If you wish to only see matches that have been scored, you can add “scored=1” to your query, likewise “scored=0” to see only matches that have not been scored.
I’ll also tag @nallen01 so that he is aware of this minor documentation issue.
For all of the queries, you use “team” for the team number, and for the returns from Rankings, Season Rankings, Awards, and Skills, you use “team” for the team number as well, but for the return on Teams, you use “number” instead.
Is this an error? Is there a deliberate reason for this?
If it is an error, is it in too wide of use as it stands to modify it now?