[Ros-kinect] Facial Recognition w/ Kinect

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[Ros-kinect] Facial Recognition w/ Kinect

David Hinkle
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- David

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
Hi David,
This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in the ROS space.
I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by Servin Emami.
Below is the link to his site that describes how to do this.
Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.

So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to and collaborate with going forward so thanks for starting the thread.
Hopefully others will jump on this thread as well to share what they know or have found.

Once the robot knows who it is talking to, that opens a lot of doors for behavior options!

http://www.shervinemami.co.cc/faceRecognition.html

Best Regards,
-Scott



On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email]> wrote:
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- Davide

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

rusu
Administrator
Scott,

If you want to do "recognition", something like the VFH descriptor might help you, especially if you can use the face
detection as a segmentation step. See http://www.ros.org/wiki/vfh_cluster_classifier for more information. I'd be more
than happy to help if anyone wants to work on a project like that.

Cheers,
Radu.
--
http://pointclouds.org

On 02/15/2011 11:39 AM, scott bell wrote:

> Hi David,
> This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in
> the ROS space.
> I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
> Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by
> Servin Emami.
> Below is the link to his site that describes how to do this.
> Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
> The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into
> view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.
>
> So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to
> and collaborate with going forward so thanks for starting the thread.
> Hopefully others will jump on this thread as well to share what they know or have found.
>
> Once the robot knows who it is talking to, that opens a lot of doors for behavior options!
>
> http://www.shervinemami.co.cc/faceRecognition.html
>
> Best Regards,
> -Scott
>
>
>
> On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email] <mailto:[hidden email]>> wrote:
>
>     So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding
>     facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with
>     that.   Is there a particular library I should be looking at?
>
>     The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.
>
>     - Davide
>
>     _______________________________________________
>     Ros-kinect mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://code.ros.org/mailman/listinfo/ros-kinect
>
>
>
>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

Patrick Goebel
In reply to this post by bellscotcv
Hi Scott,

Sounds like a cool project!  I wonder if some of the OpenNI stuff could be used to initially segment a head out of the background by first getting the skeleton (or partial skeleton).  Although I guess you wouldn't want to depend on everyone having to assume the "H" position to get the tracker's attention in the first place...

Patrick Goebel
Behavioral Sciences
Stanford University

On 2/15/2011 11:39 AM, scott bell wrote:
Hi David,
This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in the ROS space.
I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by Servin Emami.
Below is the link to his site that describes how to do this.
Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.

So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to and collaborate with going forward so thanks for starting the thread.
Hopefully others will jump on this thread as well to share what they know or have found.

Once the robot knows who it is talking to, that opens a lot of doors for behavior options!

http://www.shervinemami.co.cc/faceRecognition.html

Best Regards,
-Scott



On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email]> wrote:
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- Davide

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________ Ros-kinect mailing list [hidden email] https://code.ros.org/mailman/listinfo/ros-kinect

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

Caroline Pantofaru
There has been previous work on 3D face recognition from laser scans that should be applicable to the kinect.

I would suggest starting here: http://iris.usc.edu/Vision-Notes/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics, or more specifically here: http://iris.usc.edu/Vision-Notes/bibliography/people888.html#Face%20Recognition%20Systems

Another good portal: http://www.face-rec.org/general-info/


On Tue, Feb 15, 2011 at 1:18 PM, Patrick Goebel <[hidden email]> wrote:
Hi Scott,

Sounds like a cool project!  I wonder if some of the OpenNI stuff could be used to initially segment a head out of the background by first getting the skeleton (or partial skeleton).  Although I guess you wouldn't want to depend on everyone having to assume the "H" position to get the tracker's attention in the first place...

Patrick Goebel
Behavioral Sciences
Stanford University


On 2/15/2011 11:39 AM, scott bell wrote:
Hi David,
This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in the ROS space.
I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by Servin Emami.
Below is the link to his site that describes how to do this.
Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.

So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to and collaborate with going forward so thanks for starting the thread.
Hopefully others will jump on this thread as well to share what they know or have found.

Once the robot knows who it is talking to, that opens a lot of doors for behavior options!

http://www.shervinemami.co.cc/faceRecognition.html

Best Regards,
-Scott



On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email]> wrote:
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- Davide

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________ Ros-kinect mailing list [hidden email] https://code.ros.org/mailman/listinfo/ros-kinect

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
Thanks Caroline for the additional information and links.
I'll do some reading from the postings you and Radu have suggested and see what we can make of it.
Seems like all the pieces are there. We just need to stick them together.
To piggy back off of Radu's suggestion, one potential way to attack this would be to use basic face detection code (like from Shervin's work which I have played with before) then use that as a region of interest to crop the Kinect point cloud data down to a persons head.
Once you have that, I am guessing you could use it to then feed into Radu's vfh_cluster_classifier for training and recognition.
Lots of homework to do. But if you and Radu think this would be a good way to start, I can try to create a ros package that wrappers the facial detection code and post it somewhere on the ROS site or attach it to this thread.
Thanks again for the interest.
-Scott

P.S. I e-mailed Shervin Amami to see if he had tried to use the Kinect yet to extend his facial recognition work. He replied back saying he had not, but he very much gave his blessing to have us try and mod his code ala the Kinect and ROS to see what could be done. So it was nice to hear from him as he has done some good work in this area as well.
-S


On Tue, Feb 15, 2011 at 2:37 PM, Caroline Pantofaru <[hidden email]> wrote:
There has been previous work on 3D face recognition from laser scans that should be applicable to the kinect.
I would suggest starting here: http://iris.usc.edu/Vision-Notes/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics, or more specifically here: http://iris.usc.edu/Vision-Notes/bibliography/people888.html#Face%20Recognition%20Systems

Another good portal: http://www.face-rec.org/general-info/



On Tue, Feb 15, 2011 at 1:18 PM, Patrick Goebel <[hidden email]> wrote:
Hi Scott,

Sounds like a cool project!  I wonder if some of the OpenNI stuff could be used to initially segment a head out of the background by first getting the skeleton (or partial skeleton).  Although I guess you wouldn't want to depend on everyone having to assume the "H" position to get the tracker's attention in the first place...

Patrick Goebel
Behavioral Sciences
Stanford University


On 2/15/2011 11:39 AM, scott bell wrote:
Hi David,
This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in the ROS space.
I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by Servin Emami.
Below is the link to his site that describes how to do this.
Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.

So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to and collaborate with going forward so thanks for starting the thread.
Hopefully others will jump on this thread as well to share what they know or have found.

Once the robot knows who it is talking to, that opens a lot of doors for behavior options!

http://www.shervinemami.co.cc/faceRecognition.html

Best Regards,
-Scott



On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email]> wrote:
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- Davide

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________ Ros-kinect mailing list [hidden email] https://code.ros.org/mailman/listinfo/ros-kinect

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

Javier Correa Villanueva
Maybe this helps: http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1794&context=infopapers

Face recognition using swissranger ToF camera

On Tue, Feb 15, 2011 at 10:49 PM, scott bell <[hidden email]> wrote:
Thanks Caroline for the additional information and links.
I'll do some reading from the postings you and Radu have suggested and see what we can make of it.
Seems like all the pieces are there. We just need to stick them together.
To piggy back off of Radu's suggestion, one potential way to attack this would be to use basic face detection code (like from Shervin's work which I have played with before) then use that as a region of interest to crop the Kinect point cloud data down to a persons head.
Once you have that, I am guessing you could use it to then feed into Radu's vfh_cluster_classifier for training and recognition.
Lots of homework to do. But if you and Radu think this would be a good way to start, I can try to create a ros package that wrappers the facial detection code and post it somewhere on the ROS site or attach it to this thread.
Thanks again for the interest.
-Scott

P.S. I e-mailed Shervin Amami to see if he had tried to use the Kinect yet to extend his facial recognition work. He replied back saying he had not, but he very much gave his blessing to have us try and mod his code ala the Kinect and ROS to see what could be done. So it was nice to hear from him as he has done some good work in this area as well.
-S



On Tue, Feb 15, 2011 at 2:37 PM, Caroline Pantofaru <[hidden email]> wrote:
There has been previous work on 3D face recognition from laser scans that should be applicable to the kinect.
I would suggest starting here: http://iris.usc.edu/Vision-Notes/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics, or more specifically here: http://iris.usc.edu/Vision-Notes/bibliography/people888.html#Face%20Recognition%20Systems

Another good portal: http://www.face-rec.org/general-info/



On Tue, Feb 15, 2011 at 1:18 PM, Patrick Goebel <[hidden email]> wrote:
Hi Scott,

Sounds like a cool project!  I wonder if some of the OpenNI stuff could be used to initially segment a head out of the background by first getting the skeleton (or partial skeleton).  Although I guess you wouldn't want to depend on everyone having to assume the "H" position to get the tracker's attention in the first place...

Patrick Goebel
Behavioral Sciences
Stanford University


On 2/15/2011 11:39 AM, scott bell wrote:
Hi David,
This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in the ROS space.
I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by Servin Emami.
Below is the link to his site that describes how to do this.
Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.

So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to and collaborate with going forward so thanks for starting the thread.
Hopefully others will jump on this thread as well to share what they know or have found.

Once the robot knows who it is talking to, that opens a lot of doors for behavior options!

http://www.shervinemami.co.cc/faceRecognition.html

Best Regards,
-Scott



On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email]> wrote:
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- Davide

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________ Ros-kinect mailing list [hidden email] https://code.ros.org/mailman/listinfo/ros-kinect

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect





--
Regards, Javier Correa

Please, don't print this e-mail if it's not needed... preserve the environment is our task...

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
Thanks Javier for the response and the article.
I gave it a read. Fairly interesting. It would be good to try with the less expensive Kinect than with the Swissranger model they used.
Do you have a link to any example code that I could leverage to try this using the kinect?

For Radu, I also read your paper. Great stuff.
Seems the most difficult part of trying to use the vfh_cluster_classifier method is capturing the initial data to build the comparison library. Since this was built on top of object recognition where they have a spinning table top to create the library of object, I am not quite sure how to work out an equivalent method for a face.

On Wed, Feb 16, 2011 at 3:26 AM, Javier Correa Villanueva <[hidden email]> wrote:
Maybe this helps: http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1794&context=infopapers

Face recognition using swissranger ToF camera


On Tue, Feb 15, 2011 at 10:49 PM, scott bell <[hidden email]> wrote:
Thanks Caroline for the additional information and links.
I'll do some reading from the postings you and Radu have suggested and see what we can make of it.
Seems like all the pieces are there. We just need to stick them together.
To piggy back off of Radu's suggestion, one potential way to attack this would be to use basic face detection code (like from Shervin's work which I have played with before) then use that as a region of interest to crop the Kinect point cloud data down to a persons head.
Once you have that, I am guessing you could use it to then feed into Radu's vfh_cluster_classifier for training and recognition.
Lots of homework to do. But if you and Radu think this would be a good way to start, I can try to create a ros package that wrappers the facial detection code and post it somewhere on the ROS site or attach it to this thread.
Thanks again for the interest.
-Scott

P.S. I e-mailed Shervin Amami to see if he had tried to use the Kinect yet to extend his facial recognition work. He replied back saying he had not, but he very much gave his blessing to have us try and mod his code ala the Kinect and ROS to see what could be done. So it was nice to hear from him as he has done some good work in this area as well.
-S



On Tue, Feb 15, 2011 at 2:37 PM, Caroline Pantofaru <[hidden email]> wrote:
There has been previous work on 3D face recognition from laser scans that should be applicable to the kinect.
I would suggest starting here: http://iris.usc.edu/Vision-Notes/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics, or more specifically here: http://iris.usc.edu/Vision-Notes/bibliography/people888.html#Face%20Recognition%20Systems

Another good portal: http://www.face-rec.org/general-info/



On Tue, Feb 15, 2011 at 1:18 PM, Patrick Goebel <[hidden email]> wrote:
Hi Scott,

Sounds like a cool project!  I wonder if some of the OpenNI stuff could be used to initially segment a head out of the background by first getting the skeleton (or partial skeleton).  Although I guess you wouldn't want to depend on everyone having to assume the "H" position to get the tracker's attention in the first place...

Patrick Goebel
Behavioral Sciences
Stanford University


On 2/15/2011 11:39 AM, scott bell wrote:
Hi David,
This is something I have been researching as well and I too have not been able to find much on it yet with the Kinect in the ROS space.
I have found threads that talk about the XBOX 360 doing some facial recognition with the kinect.
Outside the Kinect/ROS code, I have played with facial recognition with OpenCV using the really good code written by Servin Emami.
Below is the link to his site that describes how to do this.
Seems like the kinect with it's depth capabilities would offer some new ways to do facial recognition.
The Willow Garage crew got the PR21 to fetch beer and bring it to a location, and would wait for a person to come into view and take the beer, but it was only doing face detection, which is a lot easier and faster than face recognition.

So this is what little I can bring to the conversation. It is nice to have others out there like yourself to speak to and collaborate with going forward so thanks for starting the thread.
Hopefully others will jump on this thread as well to share what they know or have found.

Once the robot knows who it is talking to, that opens a lot of doors for behavior options!

http://www.shervinemami.co.cc/faceRecognition.html

Best Regards,
-Scott



On Sat, Feb 12, 2011 at 3:01 PM, David Hinkle <[hidden email]> wrote:
So I've got the open ni drivers working and I've been hacking at some of the examples.   I'm interested in adding facial recognition to the mix and I wonder if anybody could give me recommendations on a good place to start with that.   Is there a particular library I should be looking at?

The end goal is a roving robot that recognizes people and then tries to detect their identity w/ facial recognition.

- Davide

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________ Ros-kinect mailing list [hidden email] https://code.ros.org/mailman/listinfo/ros-kinect

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect





--
Regards, Javier Correa

Please, don't print this e-mail if it's not needed... preserve the environment is our task...

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

rusu
Administrator

On 02/16/2011 05:46 AM, scott bell wrote:
>
> For Radu, I also read your paper. Great stuff.
> Seems the most difficult part of trying to use the vfh_cluster_classifier method is capturing the initial data to build
> the comparison library. Since this was built on top of object recognition where they have a spinning table top to create
> the library of object, I am not quite sure how to work out an equivalent method for a face.

Oh that's easy. You can record data continuously (but do run a face detector to get a 3D blob in space first), and then
build histograms for every blob as you move around in space. Using a simple similarity measure you can remove/prune data
that is too "similar", and save the rest.

We might try this today just for kicks. :)

Cheers,
Radu.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
In reply to this post by bellscotcv
Hi Radu
I am able to pretty easily do face detection via the kinect using the built in recognition python package that comes inside the vision_opencv stack.
For those on this thread here are the steps...
roscd to openni_camera
go into the launch directory, edit the openni_kinect.launch file and add the following line to remap the default video output topic from kinect to the topic that the face recognition listens node listens on.

<remap from="/camera/rgb/image_color" to="/image"/>
 
Run the face recognition node with..
rosrun opencv_tests rosfacedetect.py (located in vision_opencv stack)

If performance lags a bit, use rosrun dynamic_reconfigure reconfigure_gui to adjust settings (I tend to use the QVGA options)

For any newbies reading this, you can use rosrun rviz rviz and add a pointcloud2 option that listens on /camera/depth/points2 topic to see the point cloud.

OK....So where does this get us?
I now have the face found and boxes to a region of interest and can visualize it.
I can capture and see the point cloud data of the larger scene.
What I want to do to feed this into your recognition flow Radu is to then use the detected face ROI to crop the point cloud down to just the points that represent the face.

...and that is where I am stuck...
Any tips from anyone on how to use ROI info to crop a point cloud image on the fly would be greatly appreciated.

And for those that are interested in facial recognition, hopefully this helps you get started.

Best Regards,
-Scott


Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

rusu
Administrator
Scott,

We've done some tests with VFH (http://www.ros.org/wiki/vfh_cluster_classifier) for face recognition and things look
really good! Let us try a few more things first to cover more angles, and then we can start brainstorming on a longer
term plan. :)

Cheers,
Radu.
--
http://pointclouds.org

On 02/24/2011 11:34 AM, bellscotcv wrote:

>
> Hi Radu
> I am able to pretty easily do face detection via the kinect using the built
> in recognition python package that comes inside the vision_opencv stack.
> For those on this thread here are the steps...
> roscd to openni_camera
> go into the launch directory, edit the openni_kinect.launch file and add the
> following line to remap the default video output topic from kinect to the
> topic that the face recognition listens node listens on.
>
> <remap from="/camera/rgb/image_color" to="/image"/>
>
> Run the face recognition node with..
> rosrun opencv_tests rosfacedetect.py (located in vision_opencv stack)
>
> If performance lags a bit, use rosrun dynamic_reconfigure reconfigure_gui to
> adjust settings (I tend to use the QVGA options)
>
> For any newbies reading this, you can use rosrun rviz rviz and add a
> pointcloud2 option that listens on /camera/depth/points2 topic to see the
> point cloud.
>
> OK....So where does this get us?
> I now have the face found and boxes to a region of interest and can
> visualize it.
> I can capture and see the point cloud data of the larger scene.
> What I want to do to feed this into your recognition flow Radu is to then
> use the detected face ROI to crop the point cloud down to just the points
> that represent the face.
>
> ...and that is where I am stuck...
> Any tips from anyone on how to use ROI info to crop a point cloud image on
> the fly would be greatly appreciated.
>
> And for those that are interested in facial recognition, hopefully this
> helps you get started.
>
> Best Regards,
> -Scott
>
>
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
Outstanding. Thanks for the update Radu.
Very exciting indeed.
I will eagerly await your update.
Best Regards,
-Scott
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

dan77062
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
Hi Dan and Jenny,
That is a very cool way to approach improving the detection. I have a post on my back wall that the face recognition often also thinks is a face and the "flatness"measure would be a great way to clear up outliers like that.
Any chance you'd be willing to post some example code somewhere so the rest of us could try to play with this approach?
-Scott
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

dan77062
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

bellscotcv
That is a great question....Maybe Radu can share his thoughts.
Radu, Dan and Jenny are willing to share some code they are working on that helps to make face detection a bit more reliable once they get a little further with it. At that point, would they just attach it to this thread to share it out, or is there a more formal place or mechanism by which to share code when people correspond like this?
-Scott

On Sun, Feb 27, 2011 at 1:42 PM, dan barry <[hidden email]> wrote:
It is very much a work in progress right now, trying to get it to work on a moving robot is a challenge,  but happy to post once it is cleaned up.
Is there a place where people are posting code for this sort of topic?

--- On Sun, 2/27/11, bellscotcv <[hidden email]> wrote:

From: bellscotcv <[hidden email]>
Subject: Re: [Ros-kinect] Facial Recognition w/ Kinect
To: [hidden email]
Date: Sunday, February 27, 2011, 1:00 PM


Hi Dan and Jenny,
That is a very cool way to approach improving the detection. I have a post
on my back wall that the face recognition often also thinks is a face and
the "flatness"measure would be a great way to clear up outliers like that.
Any chance you'd be willing to post some example code somewhere so the rest
of us could try to play with this approach?
-Scott

--
View this message in context: http://kinect-with-ros.976505.n3.nabble.com/Ros-kinect-Facial-Recognition-w-Kinect-tp2482573p2589974.html
Sent from the Kinect with ROS mailing list archive at Nabble.com.
_______________________________________________
Ros-kinect mailing list
Ros-kinect@...
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

rusu
Administrator
Guys,

Sorry for the late reply - 'been really busy with PCL core work.

The best thing to do right now is to get a meeting going on with folks that are interested in the same subject, and go
quickly over multiple approaches and then discuss potential ideas for collaboration. IRC works for us, unless time
constraints are problematic, and then we'll have to continue via e-mail.

For code sharing, it's really easy to set up a repo and dump things in it.

Cheers,
Radu.
--
http://pointclouds.org

On 02/27/2011 09:27 PM, scott bell wrote:

> That is a great question....Maybe Radu can share his thoughts.
> Radu, Dan and Jenny are willing to share some code they are working on that helps to make face detection a bit more
> reliable once they get a little further with it. At that point, would they just attach it to this thread to share it
> out, or is there a more formal place or mechanism by which to share code when people correspond like this?
> -Scott
>
> On Sun, Feb 27, 2011 at 1:42 PM, dan barry <[hidden email] <mailto:[hidden email]>> wrote:
>
>     It is very much a work in progress right now, trying to get it to work on a moving robot is a challenge,  but happy
>     to post once it is cleaned up.
>     Is there a place where people are posting code for this sort of topic?
>
>     --- On *Sun, 2/27/11, bellscotcv /<[hidden email] <mailto:[hidden email]>>/* wrote:
>
>
>         From: bellscotcv <[hidden email] <mailto:[hidden email]>>
>         Subject: Re: [Ros-kinect] Facial Recognition w/ Kinect
>         To: [hidden email] <mailto:[hidden email]>
>         Date: Sunday, February 27, 2011, 1:00 PM
>
>
>         Hi Dan and Jenny,
>         That is a very cool way to approach improving the detection. I have a post
>         on my back wall that the face recognition often also thinks is a face and
>         the "flatness"measure would be a great way to clear up outliers like that.
>         Any chance you'd be willing to post some example code somewhere so the rest
>         of us could try to play with this approach?
>         -Scott
>
>         --
>         View this message in context:
>         http://kinect-with-ros.976505.n3.nabble.com/Ros-kinect-Facial-Recognition-w-Kinect-tp2482573p2589974.html
>         Sent from the Kinect with ROS mailing list archive at Nabble.com.
>         _______________________________________________
>         Ros-kinect mailing list
>         [hidden email] <http://mc/compose?to=Ros-kinect@...>
>         https://code.ros.org/mailman/listinfo/ros-kinect
>
>
>
>     _______________________________________________
>     Ros-kinect mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://code.ros.org/mailman/listinfo/ros-kinect
>
>
>
>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Facial Recognition w/ Kinect

dforce
This post has NOT been accepted by the mailing list yet.
hi everybody,

any news on that topic? code snipets etc?

cheers df