Background
The Following is a project is concerning a VR Shooter called Pavlov VR and the casting of “league” games.
Pavlov VR is more or less Counterstrike but for VR
Project Overview (Goal)
During and Post Cast of a Game there is information that can augment and improve a watchers enjoyment and understanding while also enabling the caster to speak with certainty about the overview. The Goal is to expose and make use of this information.
Considerations / Action List
There is no Verbose logging of events from the client side app (custom servers have this option but are not allowed to be used in “league” matches) meaning to get the data sought an alternative method must be found. The Strategy I have chosen to try is using a screen reader to retrieve data and spreadsheet software to sort and visualize data.
Step 1. Read Screen in order and output to CSV
Step 2. Create a spreadsheet to process csv to output
step 3. Profit… right
Tools
Capture2Text
This software has a command line interface enabling an “easy” way of getting data from a specific area of the screen and dropping that in a file.
Excel
This will allow for the data gathered in the CSV to be utilized to make diagrams/charts/tbd
Currently Foreseen Road blocks and possible ways to overcome
- I cant code - Learn
- Pavlov VR’s Font could be problematic - Tesseract data files could be produced specify to deal with this issue, If Tesseract data is being made at some point there is other items that could be added to the capture (headshot’s, guns used, util buys/usage are but a few examples of a 2.0 of this project)
- Currently unsure of how to capture, convert, and store in an order that can be replacted.
- This is all going to take place in a VM with VGPU Passthrough as this will allow for the cast to take place and for the data to be gathered at the same time using only one set of hardware.
I will update with progress, Wish me luck and please feel free to ask questions as it can help me think about aspects of the project i may not have concidered
Update 1.
Completed operations
Installed and set up HyperV VM with VGPU support
Installed, set up and tested game and spreadsheet software
Defined rectangle area’s of the screen that hold data that is required for capture - Possible future issue is this being deployed on a pc with a different resolution screen, this has been overcome by defining the resolution being used in an excel field (divided X and Z co-ords by X and Z resolution resulting in an effective percentile for each co-ord that can be used on any resalution)
Defined the CLI string to yield required data from a given area and place it in a file as an addendum
Next Operations required
Based on the result of the following test move through and capture each of the predefined areas in an order that will be consistent,
The Initial test will be preformed once per second ish and will be looking at the win counter for both teams. When ether score increases or the combine score increases the score board should be presented and recorded.
This Is the area that is lightly to cause the most issues for me and maybe the item that is lightly to take the most time for me. i will work to have this resolved within a week.
Given the above has gone correctly the following steps will entail data manipulation within excel to sort the data to a chartable item.
Update 2.
The Problems i expected to be dealing with are not the ones vexing me currently. so i procrastinated on the next step and ended up finding out i was in for a tougher time than ordinally expected.
During a deeper test of the ocr tool i have found that its good but not perfect out of the box. the font used in parts of the ui make it difficult for the reader to know for sure, it could be to do with the colours being somewhat close.
At this stage i have 2 paths i could take.
Path 1. Capture the offending text make a font pack for its admittedly small pool (just numbers 0 - 9) train a data set based on this font file for the OCR Program.
Pro. If successful I can then also train a data set for the icons used in game enabling deeper tracking of stats.
Con. Erm well easier said than done
Path 2. Because there is linier progression through the dataset it is possible to refine the ocr tools expectations through grand and ban lists. E.G. I could poll the score knowing that this is round 5 and the score was 1 – 3 due to past data so all I am looking for is the data set of “1 2” – “3 4” rather than the full range, I assume i don’t want to only look for 2 and 4 in this case as everything starts to look like 2’s and 4’s if that’s all you know.
Pro. This helps narrow down the issue but requires more testing to deuterium if that is always the case.
Con. I could be testing myself into having to do path one regardless, I need to learn how to define data set in code to do this correctly and thus need to know how to code (problematic)