Hi guys and gals, I had a thought for a race control app, which would easily (or better cheaply!) allow hobbyists to keep lap times and race positions without having to shell out for costly race control hardware. I doubt the app would be 100% accurate but I'm sure that for a standard weekend meet, rather than a full bore competition, it would probably be pretty popular. I don't have the 1st clue about writing software (apart from fiddling with mods for PC games!) so thought I would put it out there for discussion. 

Most of the code that would be used for this app is already written and there are some awesome apps out there displaying some very cool attributes, but nothing that ticks the required boxes.

So, store images of competitors quads for image recognition, and record the starters horn/gun which then activates the race timer. I thought that a smartphone with a camera capable of 60 - 120FPS set up looking across the start finish line (align an on-screen virtual line with the physical start finish line) The high frame rate capture is activated by pixel movement (ie. a quad moving through the camera's view) the images are then used to identify which competitor it is, which point they actually cross the line, and then to generate a lap time from that data in conjunction with data from the race timer.

100mph (44 meters per second) at 60 - 120FPS would equate to 73 - 36cm of movement per frame, given other software which smooth's motion blur, this should allow for accurate image recognition given that the only images to compare are the few competitors, especially if competitors have different colour quads/LED's. Any captured images of objects crossing the line, which fall below a certain margin of software recognition, are stored for adjudication by the race director.

Any input, any codewriters out there interested?

I know that not everyone uses the same color props every time on every motor like I do (always yellow) but that might be a good way to record whom is crossing the line since it wouldn't require blur correction or LEDs. Then you also wouldn't necessarily need to take photos of each quad but instead could have the app report what color passed at what time. This would greatly simplify the coding. Of course the tradeoff is people having to be stuck with specific colors of props... and if it was someone like me possibly using colors that they don't usually use.

Good idea though! I have experience with DB Architecture and can code in SQL in just about any platform. If we used a DB over the internet we could potentially post results on a website for all to view during and after the events. On the small scale level of just getting started we could easy use basic web-hosting and grow from there.

I'd be up to helping with the DB side of that if there is someone with UI/MW experience.

(13-Apr-2017, 07:00 PM)Carl.Vegas Wrote: (...) could have the app report what color passed at what time. This would greatly simplify the coding.

Perfect, maybe add vinyl stickers to the frame/arms to reinforce/re-iterate the colour of props for the software, or just to alleviate arguments..!
(Grom - Yeah, but yellow is my prop colour..!
Carl.Vegas - No, yellow is my prop colour, and it's my phone and my app *Cartman voice* Screw you guys, I'm going home!) 

(13-Apr-2017, 07:00 PM)Carl.Vegas Wrote: Good idea though! I have experience with DB Architecture and can code in SQL in just about any platform. If we used a DB over the internet we could potentially post results on a website for all to view during and after the events. On the small scale level of just getting started we could easy use basic web-hosting and grow from there.

I'd be up to helping with the DB side of that if there is someone with UI/MW experience.

This is EXACTLY the sort of response I was hoping for, Thanks so much Carl, lets hope we can get some more of the guys here interested...

As someone who does work with software that performs recognition from images, let me say that while it might sound simple, it's actually really, really hard.

Recognising the shape of a quad from an image where the quad can be presented in any arbitrary rotation and even partially obscured by other quads is going to be something that would need a high discrimination neural network simulation. The network would need to be trained to recognise the individual craft in all the various orientations that they might appear in. Time consuming and hard to actually organise.

A mobile phone's camera also doesn't give the CPU access to each frame in real time. During video capture, image data from the sensor is fed directly to a hardware video compression chip to bring the amount of data per second down to a rate that can actually be written to flash memory without totally overloading the phone's main CPU and limited RAM.

As you've already pointed out, a quad moving at 100mph would be moving at 36cm per frame in 120fps video. Even if the kind of image recognition needed could be performed on a mobile phone's limited hardware (which I doubt), the accuracy just wouldn't be there. I'm not sure what kind of shutter speeds can be achieved during 120fps video capture, but if the shutter speed isn't fast enough, the quads will just be an unrecognisable smear in every frame.

So, in my opinion, what you're talking about is actually a hard problem. Not only in terms of the actual recognition, but also in terms of video capture limitations.

If you want a definitive answer, the man to ask would be SeByDocKy over on RCGroups. Apart from being addicted to RC stuff, his day job is an associate professor at a university in France who specialises in machine vision.

(14-Apr-2017, 09:35 AM)unseen Wrote:

1. partially obscured by other quads
   
2. A mobile phone's camera also doesn't give the CPU access to each frame in real time. 
   
3. hardware video compression chip to bring the amount of data per second down to a rate that can actually be written to flash memory without totally overloading the phone's main CPU and limited RAM. 
   
4. the quads will just be an unrecognisable smear in every frame. 
   

1. Would almost certainly be an issue... perhaps part of the solution is to build a wide and low gate that encourages side-by-side entry to avoid crashing? 
   
2. A critical issue for sure. Maybe if the phone took a photo when sensing change/movement? The latency of calling that routine on a phone might be too much to grab relevant photos
   
3. I hadn't thought about that. If it does it on photos too then the tech on phones alone is definitely not up to the task and the moment you start adding other tech to the solution you might as well just get a transponder system. 
   
4. Well... if using the prop method then the smear may solve the position issue mentioned before number one. Plus a photo finish of 2 or 3 smears at once could help to add an extra layer of human error checking in case the app gets it wrong. 
   

All in all though, you're probably right... There are a lot of unknowns that would make this sort of thing very difficult at a minimum, could be flat out blockers or impossibilities at a realistic perspective. 

(post liked not because I like that it probably wouldn't work, but because it's really informative)

(14-Apr-2017, 06:45 AM)Tom BD Bad Wrote: (Grom - Yeah, but yellow is my prop colour..!
Carl.Vegas - No, yellow is my prop colour, and it's my phone and my app  *Cartman voice* Screw you guys, I'm going home!) 

I am calling dibs on yellow now in perpetuity earned by the fact that I offered to help :p Anyone else who wants to still fly yellow when I am flying can do so but mine will always be the first yellow across the line in the official results and the other yellow will always be whomever argued with me regardless of which quad it actually was  

Doing it with cameras is (in my opinion) the wrong way to go about it.

One of the best ideas I've seen recently is a system that uses the signal strength of your VTX to detect when your quad flies past a little box that you position next to the start/finish gate. Sure, each person needs their own box, but it doesn't need a transponder or anything else adding to your craft. You can connect multiple boxes to a central system for race timing. Only 60 Euro! See: https://www.airbirds-shop.de/en/easyrace...ition.html

The software for the central race timing system is open source and runs on a Raspberry Pi!

What about using RFID tags like they put in books at the library, etc?

(14-Apr-2017, 12:32 PM)unseen Wrote: Doing it with cameras is (in my opinion) the wrong way to go about it.

One of the best ideas I've seen recently is a system that uses the signal strength of your VTX to detect when your quad flies past a little box that you position next to the start/finish gate. Sure, each person needs their own box, but it doesn't need a transponder or anything else adding to your craft. You can connect multiple boxes to a central system for race timing. Only 60 Euro! See: https://www.airbirds-shop.de/en/easyrace...ition.html

The software for the central race timing system is open source and runs on a Raspberry Pi!

That's a really good idea and isn't much more expensive than the transponders that multi GP uses. I also really like that it's open source.

---

(14-Apr-2017, 12:53 PM)sloscotty Wrote: What about using RFID tags like they put in books at the library, etc?

you know... if it's good enough to catch badguys, it should be good enough to catch who crossed the finish line right?

RFID tags are certainly another option, but I really like the VTX idea as there's nothing to add to your quad.

The little sensor box can be used on your own for practise and only needs 5V power from a USB power brick to work. It will even record your laps without needing a mobile or control system so that you can download them after the fact without needing to have your smartphone connected while it is logging.

It seems TBS have a similar device as well although it currently shows as "Out of Stock".

(14-Apr-2017, 05:53 PM)unseen Wrote: RFID tags are certainly another option, but I really like the VTX idea as there's nothing to add to your quad.

The little sensor box can be used on your own for practise and only needs 5V power from a USB power brick to work. It will even record your laps without needing a mobile or control system so that you can download them after the fact without needing to have your smartphone connected while it is logging.

It seems TBS have a similar device as well although it currently shows as "Out of Stock".

when I get to a point where I am ready to time laps I might give something like that a try. For now "1 full lap without crashing" is still an acceptable metric of success. Maybe after another 20 or so batteries I'll be there!

(14-Apr-2017, 09:35 AM)unseen Wrote: As someone who does work with software that performs recognition from images, let me say that while it might sound simple, it's actually really, really hard.

Of course unseen says 'NO' to image recognition

(15-Apr-2017, 05:44 AM)Tom BD Bad Wrote: Of course unseen says 'NO' to image recognition

Just because you're not paranoid, it doesn't mean they're not out to get you!

 funny

I thought that if the recognition focussed on colour that it wouldn't require so much from the limited CPU and ram..? Just wondering if maybe an audio signature could be recognized? Is this a brainwave or early morning stupity?

Edit to add, remember that this is just a hobby tool, I'm not looking for Tag-Heuer or Rolex to get involved!

I've never tried writing code for Android or iOS(spit) that tries to get video frames in real time. I'm not even sure if you can.

Even if that's possible, I have serious doubts as to how reliable and repeatable detecting colour would be. As to audio, I can't imagine how you'd make each quad have a unique sound that would distinguish it from other quads at all possible speeds and throttle settings. Even if you could, how would you infer the position of several quads flying at speed and separated by only a small distance by sound alone?