[Ros-kinect] Mapping image pixels to depth information in point cloud

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[Ros-kinect] Mapping image pixels to depth information in point cloud

ahelou
Hello,

I am using the kinect driver provided in the kinect_driver stack. I am interested in mapping each pixel in the /kinect/rgb/image_raw topic to the corresponding depth values in /kinect/depth/points or /kinect/rgb/points. Initially, I assumed that there would be a straightforward 1:1 mapping but apparently that isn't the case. I suspect that the answer lies in /kinect/rgb/points because rviz can overlay the depth points over the RGB image using this topic only. I've consulted the ros kinect page but I was not able to get far. BTW, I am using the PointCloud message instead of PointCloud2 because rviz freezes whenever I try to display point clouds formatted as PointCloud2.

On a side note, what is the unit used to report the depth values in the point cloud topic as well as the disparity image topic?

Thank you,
Akram

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

rusu
Administrator
Akram,

On 01/23/2011 07:50 AM, Akram Helou wrote:
> Hello,
>
> I am using the kinect driver provided in the kinect_driver stack. I am interested in mapping each pixel in the
> /kinect/rgb/image_raw topic to the corresponding depth values in /kinect/depth/points or /kinect/rgb/points. Initially,
> I assumed that there would be a straightforward 1:1 mapping but apparently that isn't the case. I suspect that the
> answer lies in /kinect/rgb/points because rviz can overlay the depth points over the RGB image using this topic only.
> I've consulted the ros kinect page but I was not able to get far. BTW, I am using the PointCloud message instead of
> PointCloud2 because rviz freezes whenever I try to display point clouds formatted as PointCloud2.

Do you happen to know whether you're using the latest version of RViz? I believe this bug got fixed.

> On a side note, what is the unit used to report the depth values in the point cloud topic as well as the disparity image
> topic?

They should be in meters.

Just out of curiosity, is there a reason why you picked the kinect stack over the ni one?

Thanks,
Radu.
--
http://pointclouds.org
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

ahelou
Radu,

1- I've updated to the latest rviz version and I am not able to display PontCloud2 information without any problems.
2- There is no particular reason why I chose the kinect driver over the ni driver. But since the ni driver is designated as "recommended" then I will switch to it.
3- I still need help with my main issue that is:
> I am interested in mapping each pixel in the
> /kinect/rgb/image_raw topic to the corresponding depth values in /kinect/depth/points or /kinect/rgb/points. Initially,
> I assumed that there would be a straightforward 1:1 mapping but apparently that isn't the case. I suspect that the
> answer lies in /kinect/rgb/points because rviz can overlay the depth points over the RGB image using this topic only.
> I've consulted the ros kinect page but I was not able to get far.

Thanks,
Akram

On Sun, Jan 23, 2011 at 9:06 PM, Radu Bogdan Rusu <[hidden email]> wrote:
Akram,

On 01/23/2011 07:50 AM, Akram Helou wrote:
> Hello,
>
> I am using the kinect driver provided in the kinect_driver stack. I am interested in mapping each pixel in the
> /kinect/rgb/image_raw topic to the corresponding depth values in /kinect/depth/points or /kinect/rgb/points. Initially,
> I assumed that there would be a straightforward 1:1 mapping but apparently that isn't the case. I suspect that the
> answer lies in /kinect/rgb/points because rviz can overlay the depth points over the RGB image using this topic only.
> I've consulted the ros kinect page but I was not able to get far. BTW, I am using the PointCloud message instead of
> PointCloud2 because rviz freezes whenever I try to display point clouds formatted as PointCloud2.

Do you happen to know whether you're using the latest version of RViz? I believe this bug got fixed.

> On a side note, what is the unit used to report the depth values in the point cloud topic as well as the disparity image
> topic?

They should be in meters.

Just out of curiosity, is there a reason why you picked the kinect stack over the ni one?

Thanks,
Radu.
--
http://pointclouds.org


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

rusu
Administrator
Akram,


On 01/23/2011 07:30 PM, Akram Helou wrote:
> Radu,
>
> 1- I've updated to the latest rviz version and I am not able to display PontCloud2 information without any problems.

"not" = "now"? Does it work better after the update or do you still experience the same problem?

> 2- There is no particular reason why I chose the kinect driver over the ni driver. But since the ni driver is designated
> as "recommended" then I will switch to it.
> 3- I still need help with my main issue that is:
>  > I am interested in mapping each pixel in the
>  > /kinect/rgb/image_raw topic to the corresponding depth values in /kinect/depth/points or /kinect/rgb/points. Initially,
>  > I assumed that there would be a straightforward 1:1 mapping but apparently that isn't the case. I suspect that the
>  > answer lies in /kinect/rgb/points because rviz can overlay the depth points over the RGB image using this topic only.
>  > I've consulted the ros kinect page but I was not able to get far.

Hmm, there is a 1-1 mapping for sure between the points2 topic and the RGB image. Try using the at (u, v) or () (u, v)
operator.

Cheers,
Radu.
--
http://pointclouds.org
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

ahelou
Radu,

Yes, "not" = "now", sorry. 

I've switched to the openni driver but I am unable to display the disparity image in rviz. I added a camera topic in rviz and then subscribed to /camera/depth/image but I don't get any images displayed. Furthermore, there are many camera topics which I am unable to select in rviz including /camera/depth/disparity. I have "fixed frame" set to /openni_depth_frame and "target frame" to <Fixed Frame>. I am using the latest version of rviz.

In contrast, when I was using the other kinect driver, I was able to display the raw rgb image, the disparity image, and the point cloud simultaneously in rviz without any problems.

Thank you,
Akram

On Sun, Jan 23, 2011 at 11:30 PM, Radu Bogdan Rusu <[hidden email]> wrote:
Akram,


On 01/23/2011 07:30 PM, Akram Helou wrote:
> Radu,
>
> 1- I've updated to the latest rviz version and I am not able to display PontCloud2 information without any problems.

"not" = "now"? Does it work better after the update or do you still experience the same problem?

> 2- There is no particular reason why I chose the kinect driver over the ni driver. But since the ni driver is designated
> as "recommended" then I will switch to it.
> 3- I still need help with my main issue that is:
>  > I am interested in mapping each pixel in the
>  > /kinect/rgb/image_raw topic to the corresponding depth values in /kinect/depth/points or /kinect/rgb/points. Initially,
>  > I assumed that there would be a straightforward 1:1 mapping but apparently that isn't the case. I suspect that the
>  > answer lies in /kinect/rgb/points because rviz can overlay the depth points over the RGB image using this topic only.
>  > I've consulted the ros kinect page but I was not able to get far.

Hmm, there is a 1-1 mapping for sure between the points2 topic and the RGB image. Try using the at (u, v) or () (u, v)
operator.

Cheers,
Radu.
--
http://pointclouds.org


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

rusu
Administrator
Akram,


On 01/24/2011 08:49 AM, Akram Helou wrote:
> Radu,
>
> Yes, "not" = "now", sorry.

Np :)

> I've switched to the openni driver but I am unable to display the disparity image in rviz. I added a camera topic in
> rviz and then subscribed to /camera/depth/image but I don't get any images displayed. Furthermore, there are many camera
> topics which I am unable to select in rviz including /camera/depth/disparity. I have "fixed frame" set to
> /openni_depth_frame and "target frame" to <Fixed Frame>. I am using the latest version of rviz.
>
> In contrast, when I was using the other kinect driver, I was able to display the raw rgb image, the disparity image, and
> the point cloud simultaneously in rviz without any problems.

Can you please try to debug to see if the driver is working at all? E.g., image_view, openni_viewer, and/or rostopic...

Cheers,
Radu.
--
http://pointclouds.org
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

ahelou
The driver seems to be partially working because:
1- I am able to display the point cloud in rviz via the the /camera/depth/points2 topic.

2- rostopic list yields the following topics:
/camera/depth/camera_info
/camera/depth/disparity
/camera/depth/image
/camera/depth/image/compressed
/camera/depth/image/theora
/camera/depth/points2
/camera/rgb/camera_info
/camera/rgb/image_color
/camera/rgb/image_color/compressed
/camera/rgb/image_color/theora
/camera/rgb/image_mono
/camera/rgb/image_mono/compressed
/camera/rgb/image_mono/theora
/camera/rgb/image_raw
/camera/rgb/image_raw/compressed
/camera/rgb/image_raw/theora
/initialpose
/move_base_simple/goal
/openni_camera/parameter_descriptions
/openni_camera/parameter_updates
/reset_time
/rosout
/rosout_agg
/tf

3- using image_view I cannot display the topics /camera/depth/disparity or /camera/depth/image but I can display /camera/rgb/image_raw for example.

Do you think I need to delete my ni stack and reinstall?

Thanks,
Akram

On Mon, Jan 24, 2011 at 12:06 PM, Radu Bogdan Rusu <[hidden email]> wrote:
Akram,


On 01/24/2011 08:49 AM, Akram Helou wrote:
> Radu,
>
> Yes, "not" = "now", sorry.

Np :)

> I've switched to the openni driver but I am unable to display the disparity image in rviz. I added a camera topic in
> rviz and then subscribed to /camera/depth/image but I don't get any images displayed. Furthermore, there are many camera
> topics which I am unable to select in rviz including /camera/depth/disparity. I have "fixed frame" set to
> /openni_depth_frame and "target frame" to <Fixed Frame>. I am using the latest version of rviz.
>
> In contrast, when I was using the other kinect driver, I was able to display the raw rgb image, the disparity image, and
> the point cloud simultaneously in rviz without any problems.

Can you please try to debug to see if the driver is working at all? E.g., image_view, openni_viewer, and/or rostopic...

Cheers,
Radu.
--
http://pointclouds.org


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

Eric Perko
On Mon, Jan 24, 2011 at 3:20 PM, Akram Helou <[hidden email]> wrote:
The driver seems to be partially working because:
1- I am able to display the point cloud in rviz via the the /camera/depth/points2 topic.

2- rostopic list yields the following topics:
/camera/depth/camera_info
/camera/depth/disparity
/camera/depth/image
/camera/depth/image/compressed
/camera/depth/image/theora
/camera/depth/points2
/camera/rgb/camera_info
/camera/rgb/image_color
/camera/rgb/image_color/compressed
/camera/rgb/image_color/theora
/camera/rgb/image_mono
/camera/rgb/image_mono/compressed
/camera/rgb/image_mono/theora
/camera/rgb/image_raw
/camera/rgb/image_raw/compressed
/camera/rgb/image_raw/theora
/initialpose
/move_base_simple/goal
/openni_camera/parameter_descriptions
/openni_camera/parameter_updates
/reset_time
/rosout
/rosout_agg
/tf

3- using image_view I cannot display the topics /camera/depth/disparity or /camera/depth/image but I can display /camera/rgb/image_raw for example.

Do you get anything from 'rostopic hz /camera/depth/disparity' or 'rostopic hz /camera/depth/image'? Do you just get black images or no image at all?
 

Do you think I need to delete my ni stack and reinstall?

Thanks,
Akram

On Mon, Jan 24, 2011 at 12:06 PM, Radu Bogdan Rusu <[hidden email]> wrote:
Akram,


On 01/24/2011 08:49 AM, Akram Helou wrote:
> Radu,
>
> Yes, "not" = "now", sorry.

Np :)

> I've switched to the openni driver but I am unable to display the disparity image in rviz. I added a camera topic in
> rviz and then subscribed to /camera/depth/image but I don't get any images displayed. Furthermore, there are many camera
> topics which I am unable to select in rviz including /camera/depth/disparity. I have "fixed frame" set to
> /openni_depth_frame and "target frame" to <Fixed Frame>. I am using the latest version of rviz.
>
> In contrast, when I was using the other kinect driver, I was able to display the raw rgb image, the disparity image, and
> the point cloud simultaneously in rviz without any problems.

Can you please try to debug to see if the driver is working at all? E.g., image_view, openni_viewer, and/or rostopic...

Cheers,
Radu.
--
http://pointclouds.org


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

ahelou
Eric,

Both  /camera/depth/disparity' or 'rostopic hz /camera/depth/image' are showing activity at rate of about 30 Hz.

In rviz /camera/depth/image yields no image at all. However image_view shows a a VERY dark disparity image which I initially though was completely black. On the other hand, /camera/depth/disparity doesn't show in the list of topics fetched by rviz and image_view simply opens an empty frame (it doesn't show anything).

Thanks,
Akram
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

Kei Okada
You need to use disparity_view to show /camera/depth/disparity
as described in
https://code.ros.org/lurker/message/20101215.180643.f04fbd83.ja.html

rosrun disparity_view disparity_view disparity:=/camera/depth/disparity
will works.

On Tue, Jan 25, 2011 at 8:41 AM, ahelou <[hidden email]> wrote:

>
> Eric,
>
> Both  /camera/depth/disparity' or 'rostopic hz /camera/depth/image' are
> showing activity at rate of about 30 Hz.
>
> In rviz /camera/depth/image yields no image at all. However image_view shows
> a a VERY dark disparity image which I initially though was completely black.
> On the other hand, /camera/depth/disparity doesn't show in the list of
> topics fetched by rviz and image_view simply opens an empty frame (it
> doesn't show anything).
>
> Thanks,
> Akram
> --
> View this message in context: http://kinect-with-ros.976505.n3.nabble.com/Ros-kinect-Mapping-image-pixels-to-depth-information-in-point-cloud-tp2314002p2324514.html
> Sent from the Kinect with ROS mailing list archive at Nabble.com.
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Mapping image pixels to depth information in point cloud

Noussa
In reply to this post by ahelou
Akram,

I've just begun to use kinect with ROS, to get XYZRGB clouds and after that use PCL.
The first task is to map between the points2 topic and the RGB image, as you have done.
Can you explain to me please step by step what did you done exactly.

Thanks,
Noussa