[Ros-kinect] Calibration file format

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Ros-kinect] Calibration file format

Nicolas Burrus
Hi,

Investigating a little bit more, I think there is an issue with the
calibration_rgb.yaml file. As I understand it, the projection matrix
field is used for the rectification phase, as the newCameraMatrix
parameter of opencv initUndistortRectifyMap. But since we are not
actually doing a stereo rectification (in the sense of opencv
stereoRectify), this matrix should be kept as the original intrinsics
matrix.

In the file I sent, I had set projection_matrix = intrinsics * [R T]
and this leads to shifted undistorted color images (testing with the
imgproc node).

Using projection_matrix = intrinsics the un-distortion is fine, but
then we lose the information about the stereo transformation R and T
and further modules may need it. So, how should we include it along
with the images in ROS?

Cheers,
Nicolas

On Sun, Nov 21, 2010 at 1:55 PM, Nicolas Burrus <[hidden email]> wrote:

> Hi,
>
> Giving a quick look at
> rviz/src/rviz/default_plugin/camera_display.cpp, I think the clip
> effect you are getting is mostly related to its visualization, and
> seems to rather be an additional black border than pixel removal.
>
> I don't really understand all the details of the camera plugin, but as
> far as a understand it if you visualize the image_raw topic it does
> not do any rectification using the distortion coefficients, it just
> makes sure that the whole image is visible in the Ogre window.
>
> Someone can confirm this?
>
> Cheers,
> Nicolas
>
> On Sat, Nov 20, 2010 at 9:55 PM, Ivan Dryanovski
> <[hidden email]> wrote:
>> On Sat, Nov 20, 2010 at 2:59 PM, Nicolas Burrus <[hidden email]> wrote:
>>> (copy of the post on openkinect ML)
>>>
>>> All right, here are the calibration files in .yaml format. I hope this
>>> will be fully compatible with ROS, but I cannot run ROS with kinect
>>> and test it right now, I don't have a Kinect at home.
>>>
>>> I also updated some details about the simple (but only semi-automatic)
>>> calibration procedure:
>>> http://nicolas.burrus.name/index.php/Research/Kinect
>>>
>>> Please let me know if I did not understand correctly the file format,
>>> this is my first contact with ROS.
>>>
>>> Cheers,
>>> Nicolas
>>>
>>> On Fri, Nov 19, 2010 at 6:21 PM, Radu Bogdan Rusu <[hidden email]> wrote:
>>>> Nicolas, good job! Can you integrate these parameters into the
>>>> calibration.yaml file and try the ROS driver? :)
>>>>
>>>> Cheers,
>>>> Radu.
>>>>
>>>>
>>>> On 11/19/2010 03:33 AM, Nicolas Burrus wrote:
>>>>>
>>>>> I'm curious to see how much accuracy you can get. On my side, I could
>>>>> improve the results I already posted using a bigger pattern, but with
>>>>> the stereo-like calibration I still get an estimated distance between
>>>>> the two sensors of about 2cm, where as the distance we can measure
>>>>> with a ruler on the real kinect is about 2.4cm, which is quite a
>>>>> significant difference. I guess this is related to the relative
>>>>> inaccuracy of edges in the depth frame.
>>>>>
>>>>> Visual results are however pretty acceptable, especially is you look
>>>>> at the small yellow cup on this video:
>>>>> http://www.youtube.com/watch?v=BVxKvZKKpds
>>>>>
>>>>> Here are my latest parameters:
>>>>> Color
>>>>> fx 5.2921508098293293e+02
>>>>> fy 5.2556393630057437e+02
>>>>> cx 3.2894272028759258e+02
>>>>> cy 2.6748068171871557e+02
>>>>> k1 2.6451622333009589e-01
>>>>> k2 -8.3990749424620825e-01
>>>>> p1 -1.9922302173693159e-03
>>>>> p2 1.4371995932897616e-03
>>>>> k3 9.1192465078713847e-01
>>>>>
>>>>> Depth
>>>>> fx 5.9421434211923247e+02
>>>>> fy 5.9104053696870778e+02
>>>>> cx 3.3930780975300314e+02
>>>>> cy 2.4273913761751615e+02
>>>>> k1 -2.6386489753128833e-01
>>>>> k2 9.9966832163729757e-01
>>>>> p1 -7.6275862143610667e-04
>>>>> p2 5.0350940090814270e-03
>>>>> k3 -1.3053628089976321e+00
>>>>>
>>>>> Relative transform between the sensors (rotation matrix and
>>>>> translation in meters, depth image is the reference)
>>>>> R
>>>>> [ 9.9984628826577793e-01, 1.2635359098409581e-03,
>>>>>  -1.7487233004436643e-02, -1.4779096108364480e-03,
>>>>>   9.9992385683542895e-01, -1.2251380107679535e-02,
>>>>>   1.7470421412464927e-02, 1.2275341476520762e-02,
>>>>>   9.9977202419716948e-01 ]
>>>>>
>>>>> T
>>>>> [ 1.9985242312092553e-02, -7.4423738761617583e-04,
>>>>>   -1.0916736334336222e-02 ]
>>>>>
>>>>> Cheers,
>>>>> Nicolas Burrus
>>>>>
>>>>> On Fri, Nov 19, 2010 at 8:32 AM, Patrick Mihelich
>>>>> <[hidden email]>  wrote:
>>>>>>
>>>>>> My own arts and crafts work is mostly done, so hopefully tomorrow I'll
>>>>>> have
>>>>>> some depth-RGB calibration results. I'm using a calibration pattern based
>>>>>> on
>>>>>> circles, as suggested by Radu. This should be more stable than the
>>>>>> checkerboard corners, as you can look for the centroid of whatever blob
>>>>>> the
>>>>>> depth camera sees. The code isn't finished yet, but should be releasable
>>>>>> soon.
>>>>>>
>>>>>> In the mean time, I've added "tilt" to kinect_camera's set of
>>>>>> reconfigurable
>>>>>> parameters. This is probably not the ideal ROS interface, but for now
>>>>>> it's a
>>>>>> convenient way to tilt the camera up and down. Don't wear out your motor
>>>>>> though :).
>>>>>>
>>>>>> Cheers,
>>>>>> Patrick
>>>>>>
>>>>>> On Thu, Nov 18, 2010 at 4:17 PM, Ivan
>>>>>> Dryanovski<[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Git updated - fixed bug where camera frame was not being set
>>>>>>> correctly. added option to set camera_info_url to auto
>>>>>>>
>>>>>>> Ivan
>>>>>>> _______________________________________________
>>>>>>> Ros-kinect mailing list
>>>>>>> [hidden email]
>>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ros-kinect mailing list
>>>>>> [hidden email]
>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Ros-kinect mailing list
>>>>> [hidden email]
>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ros-kinect mailing list
>>> [hidden email]
>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>
>>>
>>
>>
>> Hi,
>>
>> I plugged in the calibration files that Nicolas provided. I'm
>> attaching a screenshot of the rectified images that I'm getting with
>> ROS. The images seem to be clipped - is that normal, or does it point
>> to a wrong calibration? I wonder if each individual kinect is
>> significantly different from the other ones.
>>
>> I will push the drivers into git, and perhaps other people can see
>> what results they get.
>>
>> Ivan
>>
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
>>
>>
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Calibration file format

Hordur Johannsson
I tested the calibration parameters from Nicolas to calculate a lookup in
the color image. I then replaced the
          // Fill in RGB
          pstep[d++] = 0;

With the corresponding color.

I've attached the colored point cloud from rviz. The scene is in the 1-4m range
the back wall is about 4m away. The color aligns pretty well for the nearby objects
but is a bit of for the objects farther away, e.g. the box on the backwall is off
by around ~10cm

What is the plan with the color in the point cloud?
Would it make sense to have the color calculation optional, e.g. have topics
  /kinect/points2
  /kinect/points2_color
  ...

Hordur

On Sun, Nov 21, 2010 at 8:08 AM, Nicolas Burrus <[hidden email]> wrote:
Hi,

Investigating a little bit more, I think there is an issue with the
calibration_rgb.yaml file. As I understand it, the projection matrix
field is used for the rectification phase, as the newCameraMatrix
parameter of opencv initUndistortRectifyMap. But since we are not
actually doing a stereo rectification (in the sense of opencv
stereoRectify), this matrix should be kept as the original intrinsics
matrix.

In the file I sent, I had set projection_matrix = intrinsics * [R T]
and this leads to shifted undistorted color images (testing with the
imgproc node).

Using projection_matrix = intrinsics the un-distortion is fine, but
then we lose the information about the stereo transformation R and T
and further modules may need it. So, how should we include it along
with the images in ROS?

Cheers,
Nicolas

On Sun, Nov 21, 2010 at 1:55 PM, Nicolas Burrus <[hidden email]> wrote:
> Hi,
>
> Giving a quick look at
> rviz/src/rviz/default_plugin/camera_display.cpp, I think the clip
> effect you are getting is mostly related to its visualization, and
> seems to rather be an additional black border than pixel removal.
>
> I don't really understand all the details of the camera plugin, but as
> far as a understand it if you visualize the image_raw topic it does
> not do any rectification using the distortion coefficients, it just
> makes sure that the whole image is visible in the Ogre window.
>
> Someone can confirm this?
>
> Cheers,
> Nicolas
>
> On Sat, Nov 20, 2010 at 9:55 PM, Ivan Dryanovski
> <[hidden email]> wrote:
>> On Sat, Nov 20, 2010 at 2:59 PM, Nicolas Burrus <[hidden email]> wrote:
>>> (copy of the post on openkinect ML)
>>>
>>> All right, here are the calibration files in .yaml format. I hope this
>>> will be fully compatible with ROS, but I cannot run ROS with kinect
>>> and test it right now, I don't have a Kinect at home.
>>>
>>> I also updated some details about the simple (but only semi-automatic)
>>> calibration procedure:
>>> http://nicolas.burrus.name/index.php/Research/Kinect
>>>
>>> Please let me know if I did not understand correctly the file format,
>>> this is my first contact with ROS.
>>>
>>> Cheers,
>>> Nicolas
>>>
>>> On Fri, Nov 19, 2010 at 6:21 PM, Radu Bogdan Rusu <[hidden email]> wrote:
>>>> Nicolas, good job! Can you integrate these parameters into the
>>>> calibration.yaml file and try the ROS driver? :)
>>>>
>>>> Cheers,
>>>> Radu.
>>>>
>>>>
>>>> On 11/19/2010 03:33 AM, Nicolas Burrus wrote:
>>>>>
>>>>> I'm curious to see how much accuracy you can get. On my side, I could
>>>>> improve the results I already posted using a bigger pattern, but with
>>>>> the stereo-like calibration I still get an estimated distance between
>>>>> the two sensors of about 2cm, where as the distance we can measure
>>>>> with a ruler on the real kinect is about 2.4cm, which is quite a
>>>>> significant difference. I guess this is related to the relative
>>>>> inaccuracy of edges in the depth frame.
>>>>>
>>>>> Visual results are however pretty acceptable, especially is you look
>>>>> at the small yellow cup on this video:
>>>>> http://www.youtube.com/watch?v=BVxKvZKKpds
>>>>>
>>>>> Here are my latest parameters:
>>>>> Color
>>>>> fx 5.2921508098293293e+02
>>>>> fy 5.2556393630057437e+02
>>>>> cx 3.2894272028759258e+02
>>>>> cy 2.6748068171871557e+02
>>>>> k1 2.6451622333009589e-01
>>>>> k2 -8.3990749424620825e-01
>>>>> p1 -1.9922302173693159e-03
>>>>> p2 1.4371995932897616e-03
>>>>> k3 9.1192465078713847e-01
>>>>>
>>>>> Depth
>>>>> fx 5.9421434211923247e+02
>>>>> fy 5.9104053696870778e+02
>>>>> cx 3.3930780975300314e+02
>>>>> cy 2.4273913761751615e+02
>>>>> k1 -2.6386489753128833e-01
>>>>> k2 9.9966832163729757e-01
>>>>> p1 -7.6275862143610667e-04
>>>>> p2 5.0350940090814270e-03
>>>>> k3 -1.3053628089976321e+00
>>>>>
>>>>> Relative transform between the sensors (rotation matrix and
>>>>> translation in meters, depth image is the reference)
>>>>> R
>>>>> [ 9.9984628826577793e-01, 1.2635359098409581e-03,
>>>>>  -1.7487233004436643e-02, -1.4779096108364480e-03,
>>>>>   9.9992385683542895e-01, -1.2251380107679535e-02,
>>>>>   1.7470421412464927e-02, 1.2275341476520762e-02,
>>>>>   9.9977202419716948e-01 ]
>>>>>
>>>>> T
>>>>> [ 1.9985242312092553e-02, -7.4423738761617583e-04,
>>>>>   -1.0916736334336222e-02 ]
>>>>>
>>>>> Cheers,
>>>>> Nicolas Burrus
>>>>>
>>>>> On Fri, Nov 19, 2010 at 8:32 AM, Patrick Mihelich
>>>>> <[hidden email]>  wrote:
>>>>>>
>>>>>> My own arts and crafts work is mostly done, so hopefully tomorrow I'll
>>>>>> have
>>>>>> some depth-RGB calibration results. I'm using a calibration pattern based
>>>>>> on
>>>>>> circles, as suggested by Radu. This should be more stable than the
>>>>>> checkerboard corners, as you can look for the centroid of whatever blob
>>>>>> the
>>>>>> depth camera sees. The code isn't finished yet, but should be releasable
>>>>>> soon.
>>>>>>
>>>>>> In the mean time, I've added "tilt" to kinect_camera's set of
>>>>>> reconfigurable
>>>>>> parameters. This is probably not the ideal ROS interface, but for now
>>>>>> it's a
>>>>>> convenient way to tilt the camera up and down. Don't wear out your motor
>>>>>> though :).
>>>>>>
>>>>>> Cheers,
>>>>>> Patrick
>>>>>>
>>>>>> On Thu, Nov 18, 2010 at 4:17 PM, Ivan
>>>>>> Dryanovski<[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Git updated - fixed bug where camera frame was not being set
>>>>>>> correctly. added option to set camera_info_url to auto
>>>>>>>
>>>>>>> Ivan
>>>>>>> _______________________________________________
>>>>>>> Ros-kinect mailing list
>>>>>>> [hidden email]
>>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ros-kinect mailing list
>>>>>> [hidden email]
>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Ros-kinect mailing list
>>>>> [hidden email]
>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ros-kinect mailing list
>>> [hidden email]
>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>>
>>>
>>
>>
>> Hi,
>>
>> I plugged in the calibration files that Nicolas provided. I'm
>> attaching a screenshot of the rectified images that I'm getting with
>> ROS. The images seem to be clipped - is that normal, or does it point
>> to a wrong calibration? I wonder if each individual kinect is
>> significantly different from the other ones.
>>
>> I will push the drivers into git, and perhaps other people can see
>> what results they get.
>>
>> Ivan
>>
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
>>
>>
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect

pointcloud_colored.png (410K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Calibration file format

rusu
Administrator
Hordur,

We've been meaning to change that 0 to the real RGB color. Can you please push back the changes you made?

Cheers,
Radu.


On 11/21/2010 10:13 AM, Hörður Jóhannsson wrote:

> I tested the calibration parameters from Nicolas to calculate a lookup in
> the color image. I then replaced the
>            // Fill in RGB
>            pstep[d++] = 0;
>
> With the corresponding color.
>
> I've attached the colored point cloud from rviz. The scene is in the 1-4m range
> the back wall is about 4m away. The color aligns pretty well for the nearby objects
> but is a bit of for the objects farther away, e.g. the box on the backwall is off
> by around ~10cm
>
> What is the plan with the color in the point cloud?
> Would it make sense to have the color calculation optional, e.g. have topics
>    /kinect/points2
>    /kinect/points2_color
>    ...
>
> Hordur
>
> On Sun, Nov 21, 2010 at 8:08 AM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     Investigating a little bit more, I think there is an issue with the
>     calibration_rgb.yaml file. As I understand it, the projection matrix
>     field is used for the rectification phase, as the newCameraMatrix
>     parameter of opencv initUndistortRectifyMap. But since we are not
>     actually doing a stereo rectification (in the sense of opencv
>     stereoRectify), this matrix should be kept as the original intrinsics
>     matrix.
>
>     In the file I sent, I had set projection_matrix = intrinsics * [R T]
>     and this leads to shifted undistorted color images (testing with the
>     imgproc node).
>
>     Using projection_matrix = intrinsics the un-distortion is fine, but
>     then we lose the information about the stereo transformation R and T
>     and further modules may need it. So, how should we include it along
>     with the images in ROS?
>
>     Cheers,
>     Nicolas
>
>     On Sun, Nov 21, 2010 at 1:55 PM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>      > Hi,
>      >
>      > Giving a quick look at
>      > rviz/src/rviz/default_plugin/camera_display.cpp, I think the clip
>      > effect you are getting is mostly related to its visualization, and
>      > seems to rather be an additional black border than pixel removal.
>      >
>      > I don't really understand all the details of the camera plugin, but as
>      > far as a understand it if you visualize the image_raw topic it does
>      > not do any rectification using the distortion coefficients, it just
>      > makes sure that the whole image is visible in the Ogre window.
>      >
>      > Someone can confirm this?
>      >
>      > Cheers,
>      > Nicolas
>      >
>      > On Sat, Nov 20, 2010 at 9:55 PM, Ivan Dryanovski
>      > <[hidden email] <mailto:[hidden email]>> wrote:
>      >> On Sat, Nov 20, 2010 at 2:59 PM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>      >>> (copy of the post on openkinect ML)
>      >>>
>      >>> All right, here are the calibration files in .yaml format. I hope this
>      >>> will be fully compatible with ROS, but I cannot run ROS with kinect
>      >>> and test it right now, I don't have a Kinect at home.
>      >>>
>      >>> I also updated some details about the simple (but only semi-automatic)
>      >>> calibration procedure:
>      >>> http://nicolas.burrus.name/index.php/Research/Kinect
>      >>>
>      >>> Please let me know if I did not understand correctly the file format,
>      >>> this is my first contact with ROS.
>      >>>
>      >>> Cheers,
>      >>> Nicolas
>      >>>
>      >>> On Fri, Nov 19, 2010 at 6:21 PM, Radu Bogdan Rusu <[hidden email] <mailto:[hidden email]>> wrote:
>      >>>> Nicolas, good job! Can you integrate these parameters into the
>      >>>> calibration.yaml file and try the ROS driver? :)
>      >>>>
>      >>>> Cheers,
>      >>>> Radu.
>      >>>>
>      >>>>
>      >>>> On 11/19/2010 03:33 AM, Nicolas Burrus wrote:
>      >>>>>
>      >>>>> I'm curious to see how much accuracy you can get. On my side, I could
>      >>>>> improve the results I already posted using a bigger pattern, but with
>      >>>>> the stereo-like calibration I still get an estimated distance between
>      >>>>> the two sensors of about 2cm, where as the distance we can measure
>      >>>>> with a ruler on the real kinect is about 2.4cm, which is quite a
>      >>>>> significant difference. I guess this is related to the relative
>      >>>>> inaccuracy of edges in the depth frame.
>      >>>>>
>      >>>>> Visual results are however pretty acceptable, especially is you look
>      >>>>> at the small yellow cup on this video:
>      >>>>> http://www.youtube.com/watch?v=BVxKvZKKpds
>      >>>>>
>      >>>>> Here are my latest parameters:
>      >>>>> Color
>      >>>>> fx 5.2921508098293293e+02
>      >>>>> fy 5.2556393630057437e+02
>      >>>>> cx 3.2894272028759258e+02
>      >>>>> cy 2.6748068171871557e+02
>      >>>>> k1 2.6451622333009589e-01
>      >>>>> k2 -8.3990749424620825e-01
>      >>>>> p1 -1.9922302173693159e-03
>      >>>>> p2 1.4371995932897616e-03
>      >>>>> k3 9.1192465078713847e-01
>      >>>>>
>      >>>>> Depth
>      >>>>> fx 5.9421434211923247e+02
>      >>>>> fy 5.9104053696870778e+02
>      >>>>> cx 3.3930780975300314e+02
>      >>>>> cy 2.4273913761751615e+02
>      >>>>> k1 -2.6386489753128833e-01
>      >>>>> k2 9.9966832163729757e-01
>      >>>>> p1 -7.6275862143610667e-04
>      >>>>> p2 5.0350940090814270e-03
>      >>>>> k3 -1.3053628089976321e+00
>      >>>>>
>      >>>>> Relative transform between the sensors (rotation matrix and
>      >>>>> translation in meters, depth image is the reference)
>      >>>>> R
>      >>>>> [ 9.9984628826577793e-01, 1.2635359098409581e-03,
>      >>>>>  -1.7487233004436643e-02, -1.4779096108364480e-03,
>      >>>>>   9.9992385683542895e-01, -1.2251380107679535e-02,
>      >>>>>   1.7470421412464927e-02, 1.2275341476520762e-02,
>      >>>>>   9.9977202419716948e-01 ]
>      >>>>>
>      >>>>> T
>      >>>>> [ 1.9985242312092553e-02, -7.4423738761617583e-04,
>      >>>>>   -1.0916736334336222e-02 ]
>      >>>>>
>      >>>>> Cheers,
>      >>>>> Nicolas Burrus
>      >>>>>
>      >>>>> On Fri, Nov 19, 2010 at 8:32 AM, Patrick Mihelich
>      >>>>> <[hidden email] <mailto:[hidden email]>>  wrote:
>      >>>>>>
>      >>>>>> My own arts and crafts work is mostly done, so hopefully tomorrow I'll
>      >>>>>> have
>      >>>>>> some depth-RGB calibration results. I'm using a calibration pattern based
>      >>>>>> on
>      >>>>>> circles, as suggested by Radu. This should be more stable than the
>      >>>>>> checkerboard corners, as you can look for the centroid of whatever blob
>      >>>>>> the
>      >>>>>> depth camera sees. The code isn't finished yet, but should be releasable
>      >>>>>> soon.
>      >>>>>>
>      >>>>>> In the mean time, I've added "tilt" to kinect_camera's set of
>      >>>>>> reconfigurable
>      >>>>>> parameters. This is probably not the ideal ROS interface, but for now
>      >>>>>> it's a
>      >>>>>> convenient way to tilt the camera up and down. Don't wear out your motor
>      >>>>>> though :).
>      >>>>>>
>      >>>>>> Cheers,
>      >>>>>> Patrick
>      >>>>>>
>      >>>>>> On Thu, Nov 18, 2010 at 4:17 PM, Ivan
>      >>>>>> Dryanovski<[hidden email] <mailto:[hidden email]>>
>      >>>>>> wrote:
>      >>>>>>>
>      >>>>>>> Git updated - fixed bug where camera frame was not being set
>      >>>>>>> correctly. added option to set camera_info_url to auto
>      >>>>>>>
>      >>>>>>> Ivan
>      >>>>>>> _______________________________________________
>      >>>>>>> Ros-kinect mailing list
>      >>>>>>> [hidden email] <mailto:[hidden email]>
>      >>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>      >>>>>>
>      >>>>>>
>      >>>>>> _______________________________________________
>      >>>>>> Ros-kinect mailing list
>      >>>>>> [hidden email] <mailto:[hidden email]>
>      >>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>      >>>>>>
>      >>>>>>
>      >>>>> _______________________________________________
>      >>>>> Ros-kinect mailing list
>      >>>>> [hidden email] <mailto:[hidden email]>
>      >>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>      >>>>
>      >>>>
>      >>>
>      >>> _______________________________________________
>      >>> Ros-kinect mailing list
>      >>> [hidden email] <mailto:[hidden email]>
>      >>> https://code.ros.org/mailman/listinfo/ros-kinect
>      >>>
>      >>>
>      >>
>      >>
>      >> Hi,
>      >>
>      >> I plugged in the calibration files that Nicolas provided. I'm
>      >> attaching a screenshot of the rectified images that I'm getting with
>      >> ROS. The images seem to be clipped - is that normal, or does it point
>      >> to a wrong calibration? I wonder if each individual kinect is
>      >> significantly different from the other ones.
>      >>
>      >> I will push the drivers into git, and perhaps other people can see
>      >> what results they get.
>      >>
>      >> Ivan
>      >>
>      >> _______________________________________________
>      >> Ros-kinect mailing list
>      >> [hidden email] <mailto:[hidden email]>
>      >> https://code.ros.org/mailman/listinfo/ros-kinect
>      >>
>      >>
>      >
>     _______________________________________________
>     Ros-kinect mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://code.ros.org/mailman/listinfo/ros-kinect
>
>
>
>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Calibration file format

Patrick Mihelich
In reply to this post by Nicolas Burrus
On Sun, Nov 21, 2010 at 5:08 AM, Nicolas Burrus <[hidden email]> wrote:
Investigating a little bit more, I think there is an issue with the
calibration_rgb.yaml file. As I understand it, the projection matrix
field is used for the rectification phase, as the newCameraMatrix
parameter of opencv initUndistortRectifyMap. But since we are not
actually doing a stereo rectification (in the sense of opencv
stereoRectify), this matrix should be kept as the original intrinsics
matrix.

Thanks for the new parameters Nicolas. Your interpretation is correct. Although, I think an ideal solution might do a stereo rectification based on calibration, then adjust offsets based on the measured depths to solve the viewpoint problem.

Using projection_matrix = intrinsics the un-distortion is fine, but
then we lose the information about the stereo transformation R and T
and further modules may need it. So, how should we include it along
with the images in ROS?

http://www.ros.org/wiki/image_pipeline/CameraInfo has some more info about the calibration parameters. We don't keep the stereo transformation R and T, at least not in that form. The left and right rectification matrices R rotate the left and right images to line up row-by-row for efficient processing. Essentially we replace the original camera axes with ideal camera axes aligned with each other. The 3x3 submatrix P[1:3,1:3] is the new intrinsics matrix for the idealized camera. Then Tx = P[1,4] of the right camera is related to the distance between the cameras (baseline) by b = -Tx / fx.

So, you can get a translation T = [-Tx/fx 0 0]'. We assume that rectifying the images removes any rotation and out-of-axis translation between the idealized cameras.

Cheers,
Patrick


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Calibration file format

Dejan Pangercic
In reply to this post by rusu
Hi Radu,
did you already do the below mentioned change?
If not, Hordur, how did you do it?
cheers, D.

On Sun, Nov 21, 2010 at 7:33 PM, Radu Bogdan Rusu <[hidden email]> wrote:

> Hordur,
>
> We've been meaning to change that 0 to the real RGB color. Can you please push back the changes you made?
>
> Cheers,
> Radu.
>
>
> On 11/21/2010 10:13 AM, Hörður Jóhannsson wrote:
>> I tested the calibration parameters from Nicolas to calculate a lookup in
>> the color image. I then replaced the
>>            // Fill in RGB
>>            pstep[d++] = 0;
>>
>> With the corresponding color.
>>
>> I've attached the colored point cloud from rviz. The scene is in the 1-4m range
>> the back wall is about 4m away. The color aligns pretty well for the nearby objects
>> but is a bit of for the objects farther away, e.g. the box on the backwall is off
>> by around ~10cm
>>
>> What is the plan with the color in the point cloud?
>> Would it make sense to have the color calculation optional, e.g. have topics
>>    /kinect/points2
>>    /kinect/points2_color
>>    ...
>>
>> Hordur
>>
>> On Sun, Nov 21, 2010 at 8:08 AM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     Hi,
>>
>>     Investigating a little bit more, I think there is an issue with the
>>     calibration_rgb.yaml file. As I understand it, the projection matrix
>>     field is used for the rectification phase, as the newCameraMatrix
>>     parameter of opencv initUndistortRectifyMap. But since we are not
>>     actually doing a stereo rectification (in the sense of opencv
>>     stereoRectify), this matrix should be kept as the original intrinsics
>>     matrix.
>>
>>     In the file I sent, I had set projection_matrix = intrinsics * [R T]
>>     and this leads to shifted undistorted color images (testing with the
>>     imgproc node).
>>
>>     Using projection_matrix = intrinsics the un-distortion is fine, but
>>     then we lose the information about the stereo transformation R and T
>>     and further modules may need it. So, how should we include it along
>>     with the images in ROS?
>>
>>     Cheers,
>>     Nicolas
>>
>>     On Sun, Nov 21, 2010 at 1:55 PM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>>      > Hi,
>>      >
>>      > Giving a quick look at
>>      > rviz/src/rviz/default_plugin/camera_display.cpp, I think the clip
>>      > effect you are getting is mostly related to its visualization, and
>>      > seems to rather be an additional black border than pixel removal.
>>      >
>>      > I don't really understand all the details of the camera plugin, but as
>>      > far as a understand it if you visualize the image_raw topic it does
>>      > not do any rectification using the distortion coefficients, it just
>>      > makes sure that the whole image is visible in the Ogre window.
>>      >
>>      > Someone can confirm this?
>>      >
>>      > Cheers,
>>      > Nicolas
>>      >
>>      > On Sat, Nov 20, 2010 at 9:55 PM, Ivan Dryanovski
>>      > <[hidden email] <mailto:[hidden email]>> wrote:
>>      >> On Sat, Nov 20, 2010 at 2:59 PM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>>      >>> (copy of the post on openkinect ML)
>>      >>>
>>      >>> All right, here are the calibration files in .yaml format. I hope this
>>      >>> will be fully compatible with ROS, but I cannot run ROS with kinect
>>      >>> and test it right now, I don't have a Kinect at home.
>>      >>>
>>      >>> I also updated some details about the simple (but only semi-automatic)
>>      >>> calibration procedure:
>>      >>> http://nicolas.burrus.name/index.php/Research/Kinect
>>      >>>
>>      >>> Please let me know if I did not understand correctly the file format,
>>      >>> this is my first contact with ROS.
>>      >>>
>>      >>> Cheers,
>>      >>> Nicolas
>>      >>>
>>      >>> On Fri, Nov 19, 2010 at 6:21 PM, Radu Bogdan Rusu <[hidden email] <mailto:[hidden email]>> wrote:
>>      >>>> Nicolas, good job! Can you integrate these parameters into the
>>      >>>> calibration.yaml file and try the ROS driver? :)
>>      >>>>
>>      >>>> Cheers,
>>      >>>> Radu.
>>      >>>>
>>      >>>>
>>      >>>> On 11/19/2010 03:33 AM, Nicolas Burrus wrote:
>>      >>>>>
>>      >>>>> I'm curious to see how much accuracy you can get. On my side, I could
>>      >>>>> improve the results I already posted using a bigger pattern, but with
>>      >>>>> the stereo-like calibration I still get an estimated distance between
>>      >>>>> the two sensors of about 2cm, where as the distance we can measure
>>      >>>>> with a ruler on the real kinect is about 2.4cm, which is quite a
>>      >>>>> significant difference. I guess this is related to the relative
>>      >>>>> inaccuracy of edges in the depth frame.
>>      >>>>>
>>      >>>>> Visual results are however pretty acceptable, especially is you look
>>      >>>>> at the small yellow cup on this video:
>>      >>>>> http://www.youtube.com/watch?v=BVxKvZKKpds
>>      >>>>>
>>      >>>>> Here are my latest parameters:
>>      >>>>> Color
>>      >>>>> fx 5.2921508098293293e+02
>>      >>>>> fy 5.2556393630057437e+02
>>      >>>>> cx 3.2894272028759258e+02
>>      >>>>> cy 2.6748068171871557e+02
>>      >>>>> k1 2.6451622333009589e-01
>>      >>>>> k2 -8.3990749424620825e-01
>>      >>>>> p1 -1.9922302173693159e-03
>>      >>>>> p2 1.4371995932897616e-03
>>      >>>>> k3 9.1192465078713847e-01
>>      >>>>>
>>      >>>>> Depth
>>      >>>>> fx 5.9421434211923247e+02
>>      >>>>> fy 5.9104053696870778e+02
>>      >>>>> cx 3.3930780975300314e+02
>>      >>>>> cy 2.4273913761751615e+02
>>      >>>>> k1 -2.6386489753128833e-01
>>      >>>>> k2 9.9966832163729757e-01
>>      >>>>> p1 -7.6275862143610667e-04
>>      >>>>> p2 5.0350940090814270e-03
>>      >>>>> k3 -1.3053628089976321e+00
>>      >>>>>
>>      >>>>> Relative transform between the sensors (rotation matrix and
>>      >>>>> translation in meters, depth image is the reference)
>>      >>>>> R
>>      >>>>> [ 9.9984628826577793e-01, 1.2635359098409581e-03,
>>      >>>>>  -1.7487233004436643e-02, -1.4779096108364480e-03,
>>      >>>>>   9.9992385683542895e-01, -1.2251380107679535e-02,
>>      >>>>>   1.7470421412464927e-02, 1.2275341476520762e-02,
>>      >>>>>   9.9977202419716948e-01 ]
>>      >>>>>
>>      >>>>> T
>>      >>>>> [ 1.9985242312092553e-02, -7.4423738761617583e-04,
>>      >>>>>   -1.0916736334336222e-02 ]
>>      >>>>>
>>      >>>>> Cheers,
>>      >>>>> Nicolas Burrus
>>      >>>>>
>>      >>>>> On Fri, Nov 19, 2010 at 8:32 AM, Patrick Mihelich
>>      >>>>> <[hidden email] <mailto:[hidden email]>>  wrote:
>>      >>>>>>
>>      >>>>>> My own arts and crafts work is mostly done, so hopefully tomorrow I'll
>>      >>>>>> have
>>      >>>>>> some depth-RGB calibration results. I'm using a calibration pattern based
>>      >>>>>> on
>>      >>>>>> circles, as suggested by Radu. This should be more stable than the
>>      >>>>>> checkerboard corners, as you can look for the centroid of whatever blob
>>      >>>>>> the
>>      >>>>>> depth camera sees. The code isn't finished yet, but should be releasable
>>      >>>>>> soon.
>>      >>>>>>
>>      >>>>>> In the mean time, I've added "tilt" to kinect_camera's set of
>>      >>>>>> reconfigurable
>>      >>>>>> parameters. This is probably not the ideal ROS interface, but for now
>>      >>>>>> it's a
>>      >>>>>> convenient way to tilt the camera up and down. Don't wear out your motor
>>      >>>>>> though :).
>>      >>>>>>
>>      >>>>>> Cheers,
>>      >>>>>> Patrick
>>      >>>>>>
>>      >>>>>> On Thu, Nov 18, 2010 at 4:17 PM, Ivan
>>      >>>>>> Dryanovski<[hidden email] <mailto:[hidden email]>>
>>      >>>>>> wrote:
>>      >>>>>>>
>>      >>>>>>> Git updated - fixed bug where camera frame was not being set
>>      >>>>>>> correctly. added option to set camera_info_url to auto
>>      >>>>>>>
>>      >>>>>>> Ivan
>>      >>>>>>> _______________________________________________
>>      >>>>>>> Ros-kinect mailing list
>>      >>>>>>> [hidden email] <mailto:[hidden email]>
>>      >>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>>>>
>>      >>>>>>
>>      >>>>>> _______________________________________________
>>      >>>>>> Ros-kinect mailing list
>>      >>>>>> [hidden email] <mailto:[hidden email]>
>>      >>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>>>>
>>      >>>>>>
>>      >>>>> _______________________________________________
>>      >>>>> Ros-kinect mailing list
>>      >>>>> [hidden email] <mailto:[hidden email]>
>>      >>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>>
>>      >>>>
>>      >>>
>>      >>> _______________________________________________
>>      >>> Ros-kinect mailing list
>>      >>> [hidden email] <mailto:[hidden email]>
>>      >>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>
>>      >>>
>>      >>
>>      >>
>>      >> Hi,
>>      >>
>>      >> I plugged in the calibration files that Nicolas provided. I'm
>>      >> attaching a screenshot of the rectified images that I'm getting with
>>      >> ROS. The images seem to be clipped - is that normal, or does it point
>>      >> to a wrong calibration? I wonder if each individual kinect is
>>      >> significantly different from the other ones.
>>      >>
>>      >> I will push the drivers into git, and perhaps other people can see
>>      >> what results they get.
>>      >>
>>      >> Ivan
>>      >>
>>      >> _______________________________________________
>>      >> Ros-kinect mailing list
>>      >> [hidden email] <mailto:[hidden email]>
>>      >> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>
>>      >>
>>      >
>>     _______________________________________________
>>     Ros-kinect mailing list
>>     [hidden email] <mailto:[hidden email]>
>>     https://code.ros.org/mailman/listinfo/ros-kinect
>>
>>
>>
>>
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>



--
MSc. Dejan Pangercic
PhD Student/Researcher
Intelligent Autonomous Systems Group
Technische Universität München
Telephone: +49 (89) 289-26908
E-Mail: [hidden email]
WWW: http://ias.cs.tum.edu/people/pangercic
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Calibration file format

Tully Foote
Radu fixed this here https://github.com/ros-pkg-git/kinect/commit/5b71ce5728bf0cbc6c9ba3b9d9dd801e7ea81faa  I can use the PointCloud display in the unpatched rviz. 

Tully

On Tue, Nov 23, 2010 at 7:24 AM, Dejan Pangercic <[hidden email]> wrote:
Hi Radu,
did you already do the below mentioned change?
If not, Hordur, how did you do it?
cheers, D.

On Sun, Nov 21, 2010 at 7:33 PM, Radu Bogdan Rusu <[hidden email]> wrote:
> Hordur,
>
> We've been meaning to change that 0 to the real RGB color. Can you please push back the changes you made?
>
> Cheers,
> Radu.
>
>
> On 11/21/2010 10:13 AM, Hörður Jóhannsson wrote:
>> I tested the calibration parameters from Nicolas to calculate a lookup in
>> the color image. I then replaced the
>>            // Fill in RGB
>>            pstep[d++] = 0;
>>
>> With the corresponding color.
>>
>> I've attached the colored point cloud from rviz. The scene is in the 1-4m range
>> the back wall is about 4m away. The color aligns pretty well for the nearby objects
>> but is a bit of for the objects farther away, e.g. the box on the backwall is off
>> by around ~10cm
>>
>> What is the plan with the color in the point cloud?
>> Would it make sense to have the color calculation optional, e.g. have topics
>>    /kinect/points2
>>    /kinect/points2_color
>>    ...
>>
>> Hordur
>>
>> On Sun, Nov 21, 2010 at 8:08 AM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     Hi,
>>
>>     Investigating a little bit more, I think there is an issue with the
>>     calibration_rgb.yaml file. As I understand it, the projection matrix
>>     field is used for the rectification phase, as the newCameraMatrix
>>     parameter of opencv initUndistortRectifyMap. But since we are not
>>     actually doing a stereo rectification (in the sense of opencv
>>     stereoRectify), this matrix should be kept as the original intrinsics
>>     matrix.
>>
>>     In the file I sent, I had set projection_matrix = intrinsics * [R T]
>>     and this leads to shifted undistorted color images (testing with the
>>     imgproc node).
>>
>>     Using projection_matrix = intrinsics the un-distortion is fine, but
>>     then we lose the information about the stereo transformation R and T
>>     and further modules may need it. So, how should we include it along
>>     with the images in ROS?
>>
>>     Cheers,
>>     Nicolas
>>
>>     On Sun, Nov 21, 2010 at 1:55 PM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>>      > Hi,
>>      >
>>      > Giving a quick look at
>>      > rviz/src/rviz/default_plugin/camera_display.cpp, I think the clip
>>      > effect you are getting is mostly related to its visualization, and
>>      > seems to rather be an additional black border than pixel removal.
>>      >
>>      > I don't really understand all the details of the camera plugin, but as
>>      > far as a understand it if you visualize the image_raw topic it does
>>      > not do any rectification using the distortion coefficients, it just
>>      > makes sure that the whole image is visible in the Ogre window.
>>      >
>>      > Someone can confirm this?
>>      >
>>      > Cheers,
>>      > Nicolas
>>      >
>>      > On Sat, Nov 20, 2010 at 9:55 PM, Ivan Dryanovski
>>      > <[hidden email] <mailto:[hidden email]>> wrote:
>>      >> On Sat, Nov 20, 2010 at 2:59 PM, Nicolas Burrus <[hidden email] <mailto:[hidden email]>> wrote:
>>      >>> (copy of the post on openkinect ML)
>>      >>>
>>      >>> All right, here are the calibration files in .yaml format. I hope this
>>      >>> will be fully compatible with ROS, but I cannot run ROS with kinect
>>      >>> and test it right now, I don't have a Kinect at home.
>>      >>>
>>      >>> I also updated some details about the simple (but only semi-automatic)
>>      >>> calibration procedure:
>>      >>> http://nicolas.burrus.name/index.php/Research/Kinect
>>      >>>
>>      >>> Please let me know if I did not understand correctly the file format,
>>      >>> this is my first contact with ROS.
>>      >>>
>>      >>> Cheers,
>>      >>> Nicolas
>>      >>>
>>      >>> On Fri, Nov 19, 2010 at 6:21 PM, Radu Bogdan Rusu <[hidden email] <mailto:[hidden email]>> wrote:
>>      >>>> Nicolas, good job! Can you integrate these parameters into the
>>      >>>> calibration.yaml file and try the ROS driver? :)
>>      >>>>
>>      >>>> Cheers,
>>      >>>> Radu.
>>      >>>>
>>      >>>>
>>      >>>> On 11/19/2010 03:33 AM, Nicolas Burrus wrote:
>>      >>>>>
>>      >>>>> I'm curious to see how much accuracy you can get. On my side, I could
>>      >>>>> improve the results I already posted using a bigger pattern, but with
>>      >>>>> the stereo-like calibration I still get an estimated distance between
>>      >>>>> the two sensors of about 2cm, where as the distance we can measure
>>      >>>>> with a ruler on the real kinect is about 2.4cm, which is quite a
>>      >>>>> significant difference. I guess this is related to the relative
>>      >>>>> inaccuracy of edges in the depth frame.
>>      >>>>>
>>      >>>>> Visual results are however pretty acceptable, especially is you look
>>      >>>>> at the small yellow cup on this video:
>>      >>>>> http://www.youtube.com/watch?v=BVxKvZKKpds
>>      >>>>>
>>      >>>>> Here are my latest parameters:
>>      >>>>> Color
>>      >>>>> fx 5.2921508098293293e+02
>>      >>>>> fy 5.2556393630057437e+02
>>      >>>>> cx 3.2894272028759258e+02
>>      >>>>> cy 2.6748068171871557e+02
>>      >>>>> k1 2.6451622333009589e-01
>>      >>>>> k2 -8.3990749424620825e-01
>>      >>>>> p1 -1.9922302173693159e-03
>>      >>>>> p2 1.4371995932897616e-03
>>      >>>>> k3 9.1192465078713847e-01
>>      >>>>>
>>      >>>>> Depth
>>      >>>>> fx 5.9421434211923247e+02
>>      >>>>> fy 5.9104053696870778e+02
>>      >>>>> cx 3.3930780975300314e+02
>>      >>>>> cy 2.4273913761751615e+02
>>      >>>>> k1 -2.6386489753128833e-01
>>      >>>>> k2 9.9966832163729757e-01
>>      >>>>> p1 -7.6275862143610667e-04
>>      >>>>> p2 5.0350940090814270e-03
>>      >>>>> k3 -1.3053628089976321e+00
>>      >>>>>
>>      >>>>> Relative transform between the sensors (rotation matrix and
>>      >>>>> translation in meters, depth image is the reference)
>>      >>>>> R
>>      >>>>> [ 9.9984628826577793e-01, 1.2635359098409581e-03,
>>      >>>>>  -1.7487233004436643e-02, -1.4779096108364480e-03,
>>      >>>>>   9.9992385683542895e-01, -1.2251380107679535e-02,
>>      >>>>>   1.7470421412464927e-02, 1.2275341476520762e-02,
>>      >>>>>   9.9977202419716948e-01 ]
>>      >>>>>
>>      >>>>> T
>>      >>>>> [ 1.9985242312092553e-02, -7.4423738761617583e-04,
>>      >>>>>   -1.0916736334336222e-02 ]
>>      >>>>>
>>      >>>>> Cheers,
>>      >>>>> Nicolas Burrus
>>      >>>>>
>>      >>>>> On Fri, Nov 19, 2010 at 8:32 AM, Patrick Mihelich
>>      >>>>> <[hidden email] <mailto:[hidden email]>>  wrote:
>>      >>>>>>
>>      >>>>>> My own arts and crafts work is mostly done, so hopefully tomorrow I'll
>>      >>>>>> have
>>      >>>>>> some depth-RGB calibration results. I'm using a calibration pattern based
>>      >>>>>> on
>>      >>>>>> circles, as suggested by Radu. This should be more stable than the
>>      >>>>>> checkerboard corners, as you can look for the centroid of whatever blob
>>      >>>>>> the
>>      >>>>>> depth camera sees. The code isn't finished yet, but should be releasable
>>      >>>>>> soon.
>>      >>>>>>
>>      >>>>>> In the mean time, I've added "tilt" to kinect_camera's set of
>>      >>>>>> reconfigurable
>>      >>>>>> parameters. This is probably not the ideal ROS interface, but for now
>>      >>>>>> it's a
>>      >>>>>> convenient way to tilt the camera up and down. Don't wear out your motor
>>      >>>>>> though :).
>>      >>>>>>
>>      >>>>>> Cheers,
>>      >>>>>> Patrick
>>      >>>>>>
>>      >>>>>> On Thu, Nov 18, 2010 at 4:17 PM, Ivan
>>      >>>>>> Dryanovski<[hidden email] <mailto:[hidden email]>>
>>      >>>>>> wrote:
>>      >>>>>>>
>>      >>>>>>> Git updated - fixed bug where camera frame was not being set
>>      >>>>>>> correctly. added option to set camera_info_url to auto
>>      >>>>>>>
>>      >>>>>>> Ivan
>>      >>>>>>> _______________________________________________
>>      >>>>>>> Ros-kinect mailing list
>>      >>>>>>> [hidden email] <mailto:[hidden email]>
>>      >>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>>>>
>>      >>>>>>
>>      >>>>>> _______________________________________________
>>      >>>>>> Ros-kinect mailing list
>>      >>>>>> [hidden email] <mailto:[hidden email]>
>>      >>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>>>>
>>      >>>>>>
>>      >>>>> _______________________________________________
>>      >>>>> Ros-kinect mailing list
>>      >>>>> [hidden email] <mailto:[hidden email]>
>>      >>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>>
>>      >>>>
>>      >>>
>>      >>> _______________________________________________
>>      >>> Ros-kinect mailing list
>>      >>> [hidden email] <mailto:[hidden email]>
>>      >>> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>>
>>      >>>
>>      >>
>>      >>
>>      >> Hi,
>>      >>
>>      >> I plugged in the calibration files that Nicolas provided. I'm
>>      >> attaching a screenshot of the rectified images that I'm getting with
>>      >> ROS. The images seem to be clipped - is that normal, or does it point
>>      >> to a wrong calibration? I wonder if each individual kinect is
>>      >> significantly different from the other ones.
>>      >>
>>      >> I will push the drivers into git, and perhaps other people can see
>>      >> what results they get.
>>      >>
>>      >> Ivan
>>      >>
>>      >> _______________________________________________
>>      >> Ros-kinect mailing list
>>      >> [hidden email] <mailto:[hidden email]>
>>      >> https://code.ros.org/mailman/listinfo/ros-kinect
>>      >>
>>      >>
>>      >
>>     _______________________________________________
>>     Ros-kinect mailing list
>>     [hidden email] <mailto:[hidden email]>
>>     https://code.ros.org/mailman/listinfo/ros-kinect
>>
>>
>>
>>
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>



--
MSc. Dejan Pangercic
PhD Student/Researcher
Intelligent Autonomous Systems Group
Technische Universität München
Telephone: +49 (89) 289-26908
E-Mail: [hidden email]
WWW: http://ias.cs.tum.edu/people/pangercic
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



--
Tully Foote
Systems Engineer
Willow Garage, Inc.
[hidden email]
(650) 475-2827

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|

Re: [Ros-kinect] Calibration file format

Dejan Pangercic
Hi Tully,
I was actually talking about "We've been meaning to change that 0 to
the real RGB color." sentence.
In the code of the driver the former is namely still the case.
D.

On Tue, Nov 23, 2010 at 5:01 PM, Tully Foote <[hidden email]> wrote:

> Radu fixed this here
> https://github.com/ros-pkg-git/kinect/commit/5b71ce5728bf0cbc6c9ba3b9d9dd801e7ea81faa
> I can use the PointCloud display in the unpatched rviz.
>
> Tully
>
> On Tue, Nov 23, 2010 at 7:24 AM, Dejan Pangercic <[hidden email]>
> wrote:
>>
>> Hi Radu,
>> did you already do the below mentioned change?
>> If not, Hordur, how did you do it?
>> cheers, D.
>>
>> On Sun, Nov 21, 2010 at 7:33 PM, Radu Bogdan Rusu <[hidden email]>
>> wrote:
>> > Hordur,
>> >
>> > We've been meaning to change that 0 to the real RGB color. Can you
>> > please push back the changes you made?
>> >
>> > Cheers,
>> > Radu.
>> >
>> >
>> > On 11/21/2010 10:13 AM, Hörður Jóhannsson wrote:
>> >> I tested the calibration parameters from Nicolas to calculate a lookup
>> >> in
>> >> the color image. I then replaced the
>> >>            // Fill in RGB
>> >>            pstep[d++] = 0;
>> >>
>> >> With the corresponding color.
>> >>
>> >> I've attached the colored point cloud from rviz. The scene is in the
>> >> 1-4m range
>> >> the back wall is about 4m away. The color aligns pretty well for the
>> >> nearby objects
>> >> but is a bit of for the objects farther away, e.g. the box on the
>> >> backwall is off
>> >> by around ~10cm
>> >>
>> >> What is the plan with the color in the point cloud?
>> >> Would it make sense to have the color calculation optional, e.g. have
>> >> topics
>> >>    /kinect/points2
>> >>    /kinect/points2_color
>> >>    ...
>> >>
>> >> Hordur
>> >>
>> >> On Sun, Nov 21, 2010 at 8:08 AM, Nicolas Burrus <[hidden email]
>> >> <mailto:[hidden email]>> wrote:
>> >>
>> >>     Hi,
>> >>
>> >>     Investigating a little bit more, I think there is an issue with the
>> >>     calibration_rgb.yaml file. As I understand it, the projection
>> >> matrix
>> >>     field is used for the rectification phase, as the newCameraMatrix
>> >>     parameter of opencv initUndistortRectifyMap. But since we are not
>> >>     actually doing a stereo rectification (in the sense of opencv
>> >>     stereoRectify), this matrix should be kept as the original
>> >> intrinsics
>> >>     matrix.
>> >>
>> >>     In the file I sent, I had set projection_matrix = intrinsics * [R
>> >> T]
>> >>     and this leads to shifted undistorted color images (testing with
>> >> the
>> >>     imgproc node).
>> >>
>> >>     Using projection_matrix = intrinsics the un-distortion is fine, but
>> >>     then we lose the information about the stereo transformation R and
>> >> T
>> >>     and further modules may need it. So, how should we include it along
>> >>     with the images in ROS?
>> >>
>> >>     Cheers,
>> >>     Nicolas
>> >>
>> >>     On Sun, Nov 21, 2010 at 1:55 PM, Nicolas Burrus <[hidden email]
>> >> <mailto:[hidden email]>> wrote:
>> >>      > Hi,
>> >>      >
>> >>      > Giving a quick look at
>> >>      > rviz/src/rviz/default_plugin/camera_display.cpp, I think the
>> >> clip
>> >>      > effect you are getting is mostly related to its visualization,
>> >> and
>> >>      > seems to rather be an additional black border than pixel
>> >> removal.
>> >>      >
>> >>      > I don't really understand all the details of the camera plugin,
>> >> but as
>> >>      > far as a understand it if you visualize the image_raw topic it
>> >> does
>> >>      > not do any rectification using the distortion coefficients, it
>> >> just
>> >>      > makes sure that the whole image is visible in the Ogre window.
>> >>      >
>> >>      > Someone can confirm this?
>> >>      >
>> >>      > Cheers,
>> >>      > Nicolas
>> >>      >
>> >>      > On Sat, Nov 20, 2010 at 9:55 PM, Ivan Dryanovski
>> >>      > <[hidden email] <mailto:[hidden email]>>
>> >> wrote:
>> >>      >> On Sat, Nov 20, 2010 at 2:59 PM, Nicolas Burrus
>> >> <[hidden email] <mailto:[hidden email]>> wrote:
>> >>      >>> (copy of the post on openkinect ML)
>> >>      >>>
>> >>      >>> All right, here are the calibration files in .yaml format. I
>> >> hope this
>> >>      >>> will be fully compatible with ROS, but I cannot run ROS with
>> >> kinect
>> >>      >>> and test it right now, I don't have a Kinect at home.
>> >>      >>>
>> >>      >>> I also updated some details about the simple (but only
>> >> semi-automatic)
>> >>      >>> calibration procedure:
>> >>      >>> http://nicolas.burrus.name/index.php/Research/Kinect
>> >>      >>>
>> >>      >>> Please let me know if I did not understand correctly the file
>> >> format,
>> >>      >>> this is my first contact with ROS.
>> >>      >>>
>> >>      >>> Cheers,
>> >>      >>> Nicolas
>> >>      >>>
>> >>      >>> On Fri, Nov 19, 2010 at 6:21 PM, Radu Bogdan Rusu
>> >> <[hidden email] <mailto:[hidden email]>> wrote:
>> >>      >>>> Nicolas, good job! Can you integrate these parameters into
>> >> the
>> >>      >>>> calibration.yaml file and try the ROS driver? :)
>> >>      >>>>
>> >>      >>>> Cheers,
>> >>      >>>> Radu.
>> >>      >>>>
>> >>      >>>>
>> >>      >>>> On 11/19/2010 03:33 AM, Nicolas Burrus wrote:
>> >>      >>>>>
>> >>      >>>>> I'm curious to see how much accuracy you can get. On my
>> >> side, I could
>> >>      >>>>> improve the results I already posted using a bigger pattern,
>> >> but with
>> >>      >>>>> the stereo-like calibration I still get an estimated
>> >> distance between
>> >>      >>>>> the two sensors of about 2cm, where as the distance we can
>> >> measure
>> >>      >>>>> with a ruler on the real kinect is about 2.4cm, which is
>> >> quite a
>> >>      >>>>> significant difference. I guess this is related to the
>> >> relative
>> >>      >>>>> inaccuracy of edges in the depth frame.
>> >>      >>>>>
>> >>      >>>>> Visual results are however pretty acceptable, especially is
>> >> you look
>> >>      >>>>> at the small yellow cup on this video:
>> >>      >>>>> http://www.youtube.com/watch?v=BVxKvZKKpds
>> >>      >>>>>
>> >>      >>>>> Here are my latest parameters:
>> >>      >>>>> Color
>> >>      >>>>> fx 5.2921508098293293e+02
>> >>      >>>>> fy 5.2556393630057437e+02
>> >>      >>>>> cx 3.2894272028759258e+02
>> >>      >>>>> cy 2.6748068171871557e+02
>> >>      >>>>> k1 2.6451622333009589e-01
>> >>      >>>>> k2 -8.3990749424620825e-01
>> >>      >>>>> p1 -1.9922302173693159e-03
>> >>      >>>>> p2 1.4371995932897616e-03
>> >>      >>>>> k3 9.1192465078713847e-01
>> >>      >>>>>
>> >>      >>>>> Depth
>> >>      >>>>> fx 5.9421434211923247e+02
>> >>      >>>>> fy 5.9104053696870778e+02
>> >>      >>>>> cx 3.3930780975300314e+02
>> >>      >>>>> cy 2.4273913761751615e+02
>> >>      >>>>> k1 -2.6386489753128833e-01
>> >>      >>>>> k2 9.9966832163729757e-01
>> >>      >>>>> p1 -7.6275862143610667e-04
>> >>      >>>>> p2 5.0350940090814270e-03
>> >>      >>>>> k3 -1.3053628089976321e+00
>> >>      >>>>>
>> >>      >>>>> Relative transform between the sensors (rotation matrix and
>> >>      >>>>> translation in meters, depth image is the reference)
>> >>      >>>>> R
>> >>      >>>>> [ 9.9984628826577793e-01, 1.2635359098409581e-03,
>> >>      >>>>>  -1.7487233004436643e-02, -1.4779096108364480e-03,
>> >>      >>>>>   9.9992385683542895e-01, -1.2251380107679535e-02,
>> >>      >>>>>   1.7470421412464927e-02, 1.2275341476520762e-02,
>> >>      >>>>>   9.9977202419716948e-01 ]
>> >>      >>>>>
>> >>      >>>>> T
>> >>      >>>>> [ 1.9985242312092553e-02, -7.4423738761617583e-04,
>> >>      >>>>>   -1.0916736334336222e-02 ]
>> >>      >>>>>
>> >>      >>>>> Cheers,
>> >>      >>>>> Nicolas Burrus
>> >>      >>>>>
>> >>      >>>>> On Fri, Nov 19, 2010 at 8:32 AM, Patrick Mihelich
>> >>      >>>>> <[hidden email]
>> >> <mailto:[hidden email]>>  wrote:
>> >>      >>>>>>
>> >>      >>>>>> My own arts and crafts work is mostly done, so hopefully
>> >> tomorrow I'll
>> >>      >>>>>> have
>> >>      >>>>>> some depth-RGB calibration results. I'm using a calibration
>> >> pattern based
>> >>      >>>>>> on
>> >>      >>>>>> circles, as suggested by Radu. This should be more stable
>> >> than the
>> >>      >>>>>> checkerboard corners, as you can look for the centroid of
>> >> whatever blob
>> >>      >>>>>> the
>> >>      >>>>>> depth camera sees. The code isn't finished yet, but should
>> >> be releasable
>> >>      >>>>>> soon.
>> >>      >>>>>>
>> >>      >>>>>> In the mean time, I've added "tilt" to kinect_camera's set
>> >> of
>> >>      >>>>>> reconfigurable
>> >>      >>>>>> parameters. This is probably not the ideal ROS interface,
>> >> but for now
>> >>      >>>>>> it's a
>> >>      >>>>>> convenient way to tilt the camera up and down. Don't wear
>> >> out your motor
>> >>      >>>>>> though :).
>> >>      >>>>>>
>> >>      >>>>>> Cheers,
>> >>      >>>>>> Patrick
>> >>      >>>>>>
>> >>      >>>>>> On Thu, Nov 18, 2010 at 4:17 PM, Ivan
>> >>      >>>>>> Dryanovski<[hidden email]
>> >> <mailto:[hidden email]>>
>> >>      >>>>>> wrote:
>> >>      >>>>>>>
>> >>      >>>>>>> Git updated - fixed bug where camera frame was not being
>> >> set
>> >>      >>>>>>> correctly. added option to set camera_info_url to auto
>> >>      >>>>>>>
>> >>      >>>>>>> Ivan
>> >>      >>>>>>> _______________________________________________
>> >>      >>>>>>> Ros-kinect mailing list
>> >>      >>>>>>> [hidden email] <mailto:[hidden email]>
>> >>      >>>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>> >>      >>>>>>
>> >>      >>>>>>
>> >>      >>>>>> _______________________________________________
>> >>      >>>>>> Ros-kinect mailing list
>> >>      >>>>>> [hidden email] <mailto:[hidden email]>
>> >>      >>>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>> >>      >>>>>>
>> >>      >>>>>>
>> >>      >>>>> _______________________________________________
>> >>      >>>>> Ros-kinect mailing list
>> >>      >>>>> [hidden email] <mailto:[hidden email]>
>> >>      >>>>> https://code.ros.org/mailman/listinfo/ros-kinect
>> >>      >>>>
>> >>      >>>>
>> >>      >>>
>> >>      >>> _______________________________________________
>> >>      >>> Ros-kinect mailing list
>> >>      >>> [hidden email] <mailto:[hidden email]>
>> >>      >>> https://code.ros.org/mailman/listinfo/ros-kinect
>> >>      >>>
>> >>      >>>
>> >>      >>
>> >>      >>
>> >>      >> Hi,
>> >>      >>
>> >>      >> I plugged in the calibration files that Nicolas provided. I'm
>> >>      >> attaching a screenshot of the rectified images that I'm getting
>> >> with
>> >>      >> ROS. The images seem to be clipped - is that normal, or does it
>> >> point
>> >>      >> to a wrong calibration? I wonder if each individual kinect is
>> >>      >> significantly different from the other ones.
>> >>      >>
>> >>      >> I will push the drivers into git, and perhaps other people can
>> >> see
>> >>      >> what results they get.
>> >>      >>
>> >>      >> Ivan
>> >>      >>
>> >>      >> _______________________________________________
>> >>      >> Ros-kinect mailing list
>> >>      >> [hidden email] <mailto:[hidden email]>
>> >>      >> https://code.ros.org/mailman/listinfo/ros-kinect
>> >>      >>
>> >>      >>
>> >>      >
>> >>     _______________________________________________
>> >>     Ros-kinect mailing list
>> >>     [hidden email] <mailto:[hidden email]>
>> >>     https://code.ros.org/mailman/listinfo/ros-kinect
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Ros-kinect mailing list
>> >> [hidden email]
>> >> https://code.ros.org/mailman/listinfo/ros-kinect
>> > _______________________________________________
>> > Ros-kinect mailing list
>> > [hidden email]
>> > https://code.ros.org/mailman/listinfo/ros-kinect
>> >
>>
>>
>>
>> --
>> MSc. Dejan Pangercic
>> PhD Student/Researcher
>> Intelligent Autonomous Systems Group
>> Technische Universität München
>> Telephone: +49 (89) 289-26908
>> E-Mail: [hidden email]
>> WWW: http://ias.cs.tum.edu/people/pangercic
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
>
>
>
> --
> Tully Foote
> Systems Engineer
> Willow Garage, Inc.
> [hidden email]
> (650) 475-2827
>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>
>



--
MSc. Dejan Pangercic
PhD Student/Researcher
Intelligent Autonomous Systems Group
Technische Universität München
Telephone: +49 (89) 289-26908
E-Mail: [hidden email]
WWW: http://ias.cs.tum.edu/people/pangercic
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect