use touchless to control website content

Feb 10, 2009 at 11:30 AM
hi,

I just tried the touchless SDK and was quite happy to see an open source project using a webcam to interact with a graphical interface. Since I am quite new to dotnet and since my background is web related, I would like to ask if it would be possible to integrate touchless into a web site as client application to interact with it?
I thought of something like ActiveX or similar ..

I've done a bit of research and found nothing appropriate, yet. It would be nice to control web content without a mouse just using your hands and some predefined gestures ( or modify the application to learn your gestures and save it as a profile? ).

I appreciate every answer. Thank you in advance.

plex.
Coordinator
Feb 11, 2009 at 9:56 PM
Hi plex,

Unfortunately, I don't know much about running Touchless (or arbitrary dlls) client-side from a website.
Hopefully someone else does! Otherwise, you could build a client-side app that's run normally alongside a browser.
You can interpret Touchless data as gestures and then fire keyboard/mouse events.
You'd be able to scroll, pageup/pagedown, zoom (with ctrl-scrollwhell), etc. on most (read: non-flash) websites.

Also, flash has some webcam support, so one could theoretically implement something like Touchless in flash.

- Mike
Feb 13, 2009 at 10:11 AM
Hello michwass,

thank you for your reply. I'm sorry I could not answer immediately :(

I recently evaluated Touchless as an ActiveX control inside a web page, with some limitations ActiveX implies:

- the user must install and accept the new component and be willing to access system ressources ( taking over mouse control etc..)
-> this implies that a lot of user won't be willing to do so..

as an alternative I tried the mouse control integration of touchless, which is a good start, but offers limited precise control over the mouse ( as you have written as comment in the demo app ).

Using flash would be a nice alternative and I think I'm going to port it into the most recent flex version. The only disadvantage I saw was the performance, which is slighly below Touchless.

However, this project is still a great advancement in creating new human controllable interfaces that are a more intuitive ..
Keep on the good work and thank you again for sharing your project with us :)

- plex.

Coordinator
Feb 15, 2009 at 10:09 PM
Hmm, I'd definitely like to see an ActiveX use of Touchless if anyone else is interested.
Mouse control is at its beginning, I'm hoping we all can improve functionality soon.

Plex, I encourage you to post more info about your project here as it progresses.
Myself and others will be interested in your work, whether or not you use Touchless.

I'm glad to hear that you appreciate the project. Best regards,
- Mike
Feb 18, 2009 at 8:59 AM
Hello Michwass,

the more I am working on this project the more exciting it gets. We are currently testing the performance of wavelets for shape recognition and some other non-classical shape detection algorithms ( see belongie et. al. for a rather good approach in detecting shapes).

Another major problem is the recognition of gestures, like waving ones arms or turning hands to control something. But we are still at the beginning..
Unfortunatley I cannot reveal any source code due to my companies restrictions .. But for some a plain description of the process will suffice I think and will be no problem for my company ..

Thank you for reading :)

best regards,
plex.