How to run code at every new frame?

Oct 18, 2009 at 7:27 AM

Hi guys, this may be a basic question and my apologies early if it is (I haven’t done C#/C++ since university and I just spent 12 hours today trying to figure it out).

My code is the same as "sample" walkthrough up to making the marker and before the drawing.  I want to continuously display the X and Y coordinates in a LABEL on the FORM as the program update with each new frame of the webcam.  I thought you would put code in under:

void c_OnImageCaptured(object sender, CameraEventArgs e)

or

void m_OnChange(object sender, MarkerEventArgs e)

because these run with each new frame (or so I think).  But when I tried to enter something like:

label1.Text = x_coordinate.ToString();    where x_coordinate = e.EventData.X

it gave me a "cross thread operation not valid" error.  I'm not too sure what a thread is, but I gather each event function is a thread?  I moved the code to a new button and it works fine but it only updates when you click the button.

How do I make it update continuously with the frames? I will need to run more logic code and a serialport output write at this interval as well automatically with the frames.

 

Thanks for your help!! 

 

Oct 18, 2009 at 7:35 AM
//Here is the code:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using TouchlessLib;
using WebCamLib;
using System.Drawing.Imaging;

namespace WindowsFormsApplication1
{
    public partial class Form1 : Form
    {

        double xdouble;
        double ydouble;      

        public Form1()
        {
            InitializeComponent();
        }

        TouchlessMgr _touch = new TouchlessMgr();

        private void Form1_Load(object sender, EventArgs e)
        {
             
            foreach (Camera c in _touch.Cameras)
            {
                if (c != null)
                {
                    _touch.CurrentCamera = c;
                    c.OnImageCaptured += new EventHandler<CameraEventArgs>(c_OnImageCaptured);
                    break;
                }
            }
            pictureBox1.Paint += new PaintEventHandler(pictureBox1_Paint);
            
        }

        void pictureBox1_Paint(object sender, PaintEventArgs e)
        {
            lock (this)
            {
                if (_b != null)
                {
                    e.Graphics.DrawImageUnscaledAndClipped(_b, pictureBox1.ClientRectangle);

                }
            }
        }


        Bitmap _b;

        void c_OnImageCaptured(object sender, CameraEventArgs e)
        {
            if (button1.Enabled)
            {
                _b = e.Image;
                pictureBox1.Invalidate();
                                
            }
            
        }

        private void button1_Click(object sender, EventArgs e)
        {
            Bitmap c = _touch.CurrentCamera.GetCurrentImage();
            pictureBox1.BackgroundImage = c;
            button1.Enabled = false;
        }

        private void pictureBox1_MouseClick(object sender, MouseEventArgs e)
        {
            
            Marker m = _touch.AddMarker("marker", (Bitmap)pictureBox1.BackgroundImage, new Point (e.X, e.Y), 10);
            m.Highlight = true;
            m.OnChange += new EventHandler<MarkerEventArgs>(m_OnChange);
            button1.Enabled = true;
                   
        }

        void m_OnChange(object sender, MarkerEventArgs e)
        {

            xdouble = e.EventData.X;
            ydouble = e.EventData.Y;
                              
        }

        private void button2_Click(object sender, EventArgs e)
        {
                      
                label1.Text = "X = " + xdouble.ToString();
                label2.Text = "Y = " + ydouble.ToString();
          
        }

            
    }
    
}
Oct 18, 2009 at 1:58 PM
Edited Oct 18, 2009 at 2:00 PM

Hi Mahhari,

the location is choosed exactly right, the Exception you encounter is rather "normal":

You can only change GUI values in the thread that created the GUI element. Fortunately, MS has built in support for your problem to "switch the thread":
(The trick in in the if (InvokeRequired) line ;))

void m_OnChange(object sender, MarkerEventArgs e)
{
    this.UpdateMarkerDataInUI(e.EventData);
}

private void UpdateMarkerDataInUI(MarkerEventData data)
{
    if (this.InvokeRequired)
    {
        /* Self-Invoke in the GUI thread if necessary */
        this.BeginInvoke(new Action(
            UpdateMarkerDataInUI), new object[] { data });
    }
    else
    {
        this.label1.Text = String.Format("X = {0}", data.X);
        this.label2.Text = String.Format("Y = {0}", data.Y);
    }
}

You can also look into the TouchlessDemo project, File TouchlessDemo.cs, starting Line 259, where exactly this is done, too.

HTH, Florian

 

(edit: added reference to demo project)

Oct 18, 2009 at 6:26 PM
Edited Oct 18, 2009 at 6:30 PM

Hi eFloh,

Thanks for the reply!  I added your code to my code and I get this build error:

No overload for 'UpdateMarkerDataInUI' matches delegate 'System.Action' 

 

for line 102 which is:

this.BeginInvoke(new Action(UpdateMarkerDataInUI), new object[] { data });

What can I do to fix this? I tried a few things with no luck (most of this code is new to me), so I thought I better come to you!

 

 

Coordinator
Oct 19, 2009 at 12:34 AM
Edited Oct 19, 2009 at 12:35 AM

Try this.BeginInvoke(new Action<MarkerEventData>(UpdateMarkerDataInUI), new object[] { data });

Read more about Action delegates at: http://msdn.microsoft.com/en-us/library/018hxwa8.aspx

 See similar code in method OnImageCaptured in TouchlessDemo.cs calling UpdateFPSInUI.

Oct 19, 2009 at 2:08 AM

That worked Mike!

Thanks both of you guys!

(your touchless sdk is about to revolutionize how people interface with first person shooter games on xbox/ps3)

Oct 19, 2009 at 2:42 AM

For all my future code I want to execute with every fame update, I should have it under the else section with the this.label1.Text?

Will I need to have the this command before the normal commands as it is with label1?

for example:

this.SerialPort port1 = new SerialPort ("com11", 57600, ......);

this.port1.Open();

this.port1.Write(new byte[ ] {x_coordinate, y_coordinate}, 0,2);

this.port1.Close();

 

I internet searched the this command but couldn't find any useful information on it.

 

Thanks again!

Oct 20, 2009 at 5:37 AM

Hi again,

Maybe I'm asking the wrong question.  How can I take the marker X and Y data and bring it into another method?  If I call the method from within m_OnChange(...) I will still have the cross-thread issue right?  

(I bought a Visual 2008 C# book, but It doesn't really talk about this.)

Coordinator
Oct 20, 2009 at 6:05 AM

"this" is simply a reference to the instance of the object on which the method is being executed.

"this" keyword: http://msdn.microsoft.com/en-us/library/dk1507sz(VS.71).aspx

Here's BeginInvoke: http://msdn.microsoft.com/en-us/library/system.windows.forms.control.begininvoke.aspx

This explains BeginInvoke's use better: http://msdn.microsoft.com/en-us/library/2e08f6yc(VS.71).aspx

If you want to access the marker data from a separate thread (like a UI thread in the case of updating a label), you'll ideally employ thread-safe accessors and mutators for the shared data. Thread safety is a large topic in which I don't claim to be an expert. General programming reference guides should cover the topic sufficiently for you. Here's a short MSDN guide to Thread Synchronization in C#: http://msdn.microsoft.com/en-us/library/ms173179.aspx that goes beyond BeginInvoke.

Keep us up to date on your project; we like hearing about creative uses for Touchless SDK. (Are you using XIM360?)

Good luck, I hope we have been helpful.

- Mike

Oct 22, 2009 at 11:36 AM

Well, I think everything is said by Mike, The generic parameter was killed by my html-source editing, sorry.

But just to clarify your questions:

I personally use this whenever I access global (class member) fields or properties. This makes it clear to the reader that these are not locally declared variables and second ensures that, when someone later adds a local variable with the same name the meaning of the existing code is not changed. I think that depends a bit on your attitude whether you use it or not.

 

And in the code, the if-Part is entered when the execution is "not in the gui thread" and then calls itself in the else-part. So you have to add all code that acts with gui elements to the else part, any other code _could_ be in the if part, but its simply more clean to do everything in the else part. You could also move the if into the OnChanged-Eventhandler to clean it up further:

void m_OnChange(object sender, MarkerEventArgs e)
{
    if (this.InvokeRequired)
    {
        /* Invoke in the GUI thread if necessary */
        this.BeginInvoke(new Action<MarkerEventData>(
            this.MarkerChanged), new object[] { e.EventData });
    }
    else
    {
        this.MarkerChanged(e.EventData);
    }
}

private void MarkerChanged(MarkerEventData data)
{
   this.label1.Text = String.Format("X = {0}", data.X);
   this.label2.Text = String.Format("Y = {0}", data.Y);
}

This way, the MarkerChanged method is always called in the UI thread and you don't have the ugly invoke inside your business method.
Of Course, you could also pass the whole MarkerEventArgs object if you'd like.

What Mike said about threading is an imporant point:

You pass the MarkerEventData object as reference to another thread. No one guarantees you that the data will not be changed in the invoking (or any other) thread between the call to "BeginInvoke" and the execution of your code in the MarkerChanged method (so called race conditions). You have to ensure this by locking the fields yourself and/or using copied data.
Multithreading is not trivial to use and I suggest you to read and think well before using.

Again said, the code above DOES HAVE race conditions.

But in this case, no harm is done as the touchless lib will create a new instance of the MarkerEventArgs and MarkerEventData when thenext image is parsed. So the MarkerEventData object you hold in your hand in the GUI thread will not change any further.
This does not apply to the MarkerEventArgs.Marker object. This is a reference to the (one and only) marker objected created with AddMarker.
This sample shows a race condition changing the SmoothingEnabled marker property:

void m_OnChange(object sender, MarkerEventArgs e)
{
    e.EventMarker.SmoothingEnabled = true;
    this.BeginInvoke(new Action<Marker>(
        this.MarkerRaceCondition), new object[] { e.EventMarker });
   e.EventMarker.SmoothingEnabled = false;
}

private void MarkerRaceCondition(Marker m)
{
   System.Diagnostics.Debug.WriteLine(
       "GUI Thread smoothing info before sleep: " + m.SmoothingEnabled);
   Thread.Sleep(3000);
   System.Diagnostics.Debug.WriteLine(
       "GUI Thread smoothing info after sleep: " + m.SmoothingEnabled);
}

The SmoothingEnabled property will at least in the "after sleep" line show the value "false", before the sleep I cannot say, this really depends on the order in which the CLR will call your code.
(infact, even after the sleep, the value may still be true, but 3 seconds delay give plenty of time for the other thread to execute a single line ;))

 

To answer your question: You can copy the X and Y values into new variables and invoke your method with these copies. As they are fully under your control, no one (except you, hehe) will change them while you wait for your other thread to execute...

double x = e.EventData.X;
double y = e.EventData.Y;
this.BeginInvoke(new Action<double, double>( this.SafeSample), new object[] { x, y });

private void MarkerRaceCondition(double x, double y) { /* here you have your private x and y values, as double is a value type. */ }

(you could even directly move the e.EventData.X/Y into the object[] above, because they are value types. For refernece type members, such as the Marker object you would have to use the Clone() method or such in order to create the "private copy").

 

huh, lots of text...

Oct 23, 2009 at 5:40 PM

Sweet! that helps out a lot! I got my program working now.  I use "this." keyword for all my global variables and it does clean it up and make it easier to read - thanks for that tip. You guys are going to make a programmer out of me yet =)

I need to read up more about Threads and Threading.

Thanks again!