[Ros-kinect] kinect on ARM

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Ros-kinect] kinect on ARM

Brian Gerkey
hi,

I'm trying to get the Kinect working on an embedded computer with a
1.2GHZ ARM.  I've gotten over the various compilation hurdles (various
patches filed already, a tutorial to come later), and am now to the
point of trying to make the device actually work.  I'm using
Diamondback plus the latest openni code from hg @ kforge, all compiled
from source.  The kinect is the only USB device in use.

I'm seeing the following behavior: for a brief, variable period (5-45
seconds) after I've roslaunched `openni_camera/openni_node.launch`, I
can get data from the kinect.  I've seen live, valid point clouds and
images (both mono and color) in rviz, which is heartening.  However,
the dataflow always stops at some point.  To get it going again, I
have to kill and relaunch the driver.

When data's flowing, I see `openni_node` and `XnSensorServer`
consuming significant CPU.  After data stops, those processes, along
with the rest of the machine, are idle.  The `openni_node` is
responsive to `rosnode`, roswtf identifies no problems, and I'm able
to subscribe to all the topics offered by `openni_node`; I just don't
receive any messages.

Any thoughts?  Any debugging suggestions?  The `openni_node` is
surprisingly quiet; is there some way to put it in verbose / debug
mode or get diagnostics from it?

        brian.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Bill Morris
On Tue, 2011-03-08 at 21:30 -0800, Brian Gerkey wrote:
> hi,
>
> I'm trying to get the Kinect working on an embedded computer with a
> 1.2GHZ ARM.  I've gotten over the various compilation hurdles (various
> patches filed already, a tutorial to come later), and am now to the
> point of trying to make the device actually work.  I'm using
> Diamondback plus the latest openni code from hg @ kforge, all compiled
> from source.  The kinect is the only USB device in use.

...

> Any thoughts?  Any debugging suggestions?  The `openni_node` is
> surprisingly quiet; is there some way to put it in verbose / debug
> mode or get diagnostics from it?
>

Sounds interesting.

Have you tried the older driver libfreenect based driver? I think it
might be slightly faster.

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Kurt Konolige
And a lot less complicated...kk

On Tue, Mar 8, 2011 at 10:05 PM, Bill Morris <[hidden email]> wrote:

> On Tue, 2011-03-08 at 21:30 -0800, Brian Gerkey wrote:
>> hi,
>>
>> I'm trying to get the Kinect working on an embedded computer with a
>> 1.2GHZ ARM.  I've gotten over the various compilation hurdles (various
>> patches filed already, a tutorial to come later), and am now to the
>> point of trying to make the device actually work.  I'm using
>> Diamondback plus the latest openni code from hg @ kforge, all compiled
>> from source.  The kinect is the only USB device in use.
>
> ...
>
>> Any thoughts?  Any debugging suggestions?  The `openni_node` is
>> surprisingly quiet; is there some way to put it in verbose / debug
>> mode or get diagnostics from it?
>>
>
> Sounds interesting.
>
> Have you tried the older driver libfreenect based driver? I think it
> might be slightly faster.
>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

rusu
Administrator
In reply to this post by Bill Morris

On 03/08/2011 10:05 PM, Bill Morris wrote:
> Have you tried the older driver libfreenect based driver? I think it
> might be slightly faster.

Bill,

Have you made any comparisons? We'd be interested to see the results. We cannot improve something unless we know it's
broken. :)

Cheers,
Radu.
--
http://pointclouds.org
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Patrick Bouffard-2
ROS drivers aside, it would be a good sanity check to see if you can stream the data using freenect (maybe with one of the demo programs from that project). I'm not sure about apples-to-apples performance comparisons--in the end I was able to do what I wanted with both the freenect-based driver and the openni-based one. But I think there's a case to be made for the freenect driver's simplicity--you can easily trace back to the libusb calls.

Cheers,
Pat

On Tue, Mar 8, 2011 at 10:21 PM, Radu Bogdan Rusu <[hidden email]> wrote:

On 03/08/2011 10:05 PM, Bill Morris wrote:
> Have you tried the older driver libfreenect based driver? I think it
> might be slightly faster.

Bill,

Have you made any comparisons? We'd be interested to see the results. We cannot improve something unless we know it's
broken. :)

Cheers,
Radu.
--
http://pointclouds.org
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Bill Morris
In reply to this post by rusu
On Tue, 2011-03-08 at 22:21 -0800, Radu Bogdan Rusu wrote:
> On 03/08/2011 10:05 PM, Bill Morris wrote:
> > Have you tried the older driver libfreenect based driver? I think it
> > might be slightly faster.
>
> Bill,
>
> Have you made any comparisons? We'd be interested to see the results. We cannot improve something unless we know it's
> broken. :)

I haven't made any comparisons yet. Ivan may know more but I know at one
point we had a stripped down version of the kinect driver that discarded
large amounts of data because we were having performance problems. I
suspect there is a problem in libusb if you don't empty the buffers fast
enough.

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

rusu
Administrator

On 03/08/2011 10:52 PM, Bill Morris wrote:

> On Tue, 2011-03-08 at 22:21 -0800, Radu Bogdan Rusu wrote:
>> On 03/08/2011 10:05 PM, Bill Morris wrote:
>>> Have you tried the older driver libfreenect based driver? I think it
>>> might be slightly faster.
>>
>> Bill,
>>
>> Have you made any comparisons? We'd be interested to see the results. We cannot improve something unless we know it's
>> broken. :)
>
> I haven't made any comparisons yet. Ivan may know more but I know at one
> point we had a stripped down version of the kinect driver that discarded
> large amounts of data because we were having performance problems. I
> suspect there is a problem in libusb if you don't empty the buffers fast
> enough.

Bill,

It would be great to get some numbers (especially from interested parties), because that would allow all of us to:

  1) fix the OpenNI ROS interface - if there's anything to be fixed there;

  2) push on PrimeSense to fix their code and improve efficiency in OpenNI.

...if the numbers are unfavorable for OpenNI.


Cheers,
Radu.
--
http://pointclouds.org
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Bill Morris
On Tue, 2011-03-08 at 22:55 -0800, Radu Bogdan Rusu wrote:
> Bill,
>
> It would be great to get some numbers (especially from interested parties), because that would allow all of us to:
>
>   1) fix the OpenNI ROS interface - if there's anything to be fixed there;
>
>   2) push on PrimeSense to fix their code and improve efficiency in OpenNI.
>
> ...if the numbers are unfavorable for OpenNI.

I would like to help profile the code, but it's just a matter of time
management and the deadline for IROS is coming up.


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

futureignobel
In reply to this post by Brian Gerkey
Hi Brian,

1. What is the frequency of published signals when you use rostopic hz
/camera/... ?
2. Have you used dynamic_reconfigure to minimize processors load?
3. How much RAM does your machine have?
4. Could you please, out of curiosity, tell us what board exactly are
you using?

I've had similar experiences with PCL on ARM (with limited resources:
1GHz, 500MB RAM). To me it seemed like the time
openni_camera.openni_node.launch worked seamlessly depended strongly on
the amount of data published to topics. So it felt like when some
"buffer" was overflown openni_camera stopped publishing. When I chose
QQVGA and used XYZ_unregistered I obtained best results (25Hz frequency
for pointcloud2 topic) which caused the system to work best (I achieved
longest times before openni_camera needed rebooting).

5. Is it possible to configure openni_camera to just publish to one
topic? If so, try it out.

I've tried to install (and actually once succeeded) openni_camera on a
BeagleBoard-xM. I tried to summarize my know-how in a post here
http://answers.ros.org/question/130/openni-compilation-error-diamondback-version 
. For now, I quit those efforts and am working on algorithms, but would
like to come back to it as soon as I have them working on my PC, so I
think it would be great to find a place (Wiki?) where we could write our
experiences down for future reuse.

Tom


W dniu 09.03.2011 06:30, Brian Gerkey pisze:

> hi,
>
> I'm trying to get the Kinect working on an embedded computer with a
> 1.2GHZ ARM.  I've gotten over the various compilation hurdles (various
> patches filed already, a tutorial to come later), and am now to the
> point of trying to make the device actually work.  I'm using
> Diamondback plus the latest openni code from hg @ kforge, all compiled
> from source.  The kinect is the only USB device in use.
>
> I'm seeing the following behavior: for a brief, variable period (5-45
> seconds) after I've roslaunched `openni_camera/openni_node.launch`, I
> can get data from the kinect.  I've seen live, valid point clouds and
> images (both mono and color) in rviz, which is heartening.  However,
> the dataflow always stops at some point.  To get it going again, I
> have to kill and relaunch the driver.
>
> When data's flowing, I see `openni_node` and `XnSensorServer`
> consuming significant CPU.  After data stops, those processes, along
> with the rest of the machine, are idle.  The `openni_node` is
> responsive to `rosnode`, roswtf identifies no problems, and I'm able
> to subscribe to all the topics offered by `openni_node`; I just don't
> receive any messages.
>
> Any thoughts?  Any debugging suggestions?  The `openni_node` is
> surprisingly quiet; is there some way to put it in verbose / debug
> mode or get diagnostics from it?
>
> brian.
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Brian Gerkey
On Wed, Mar 9, 2011 at 12:05 AM, futureignobel <[hidden email]> wrote:
> 1. What is the frequency of published signals when you use rostopic hz
> /camera/... ?

There's a lot of variation in the initial rate, but it always goes to
zero.  Here's a representative trace:

$ rostopic hz /camera/depth/points
subscribed to [/camera/depth/points]
average rate: 6.452
        min: 0.092s max: 0.428s std dev: 0.12204s window: 7
no new messages
no new messages

I get a similar result for any /camera/* topic.  And after one topic
stops, all topics stop.  At that point, openni_node and XnSensorServer
are idle (but still responsive to ROS XMLRPC, including topic
subscriptions).

> 2. Have you used dynamic_reconfigure to minimize processors load?

I played with it a bit, and also tried changing the params in the
launch file.  I never produced a significantly different result.  What
parameter setting would minimize processor load?

> 3. How much RAM does your machine have?

512MB

> 4. Could you please, out of curiosity, tell us what board exactly are
> you using?

SheevaPlug: http://en.wikipedia.org/wiki/SheevaPlug

> I've had similar experiences with PCL on ARM (with limited resources:
> 1GHz, 500MB RAM). To me it seemed like the time
> openni_camera.openni_node.launch worked seamlessly depended strongly on
> the amount of data published to topics. So it felt like when some
> "buffer" was overflown openni_camera stopped publishing. When I chose
> QQVGA and used XYZ_unregistered I obtained best results (25Hz frequency
> for pointcloud2 topic) which caused the system to work best (I achieved
> longest times before openni_camera needed rebooting).

That sounds exactly like what I'm seeing.  It feels like openni_node
is somehow stateful; there's a qualitative change between it works and
when it doesn't.  I like your hypothesis about a buffer filling up,
but don't have any direct evidence of it.

It would help a lot to get some visibility into what the driver is
doing (talking to the device, decoding the data, etc.).  Any hints on
getting verbose / debug output?

> 5. Is it possible to configure openni_camera to just publish to one
> topic? If so, try it out.

Well, when I'm only subscribing to one topic, I would expect the node
to only be doing the work necessary to produce the data for that
topic.

> I've tried to install (and actually once succeeded) openni_camera on a
> BeagleBoard-xM. I tried to summarize my know-how in a post here
> http://answers.ros.org/question/130/openni-compilation-error-diamondback-version
> . For now, I quit those efforts and am working on algorithms, but would
> like to come back to it as soon as I have them working on my PC, so I
> think it would be great to find a place (Wiki?) where we could write our
> experiences down for future reuse.

Good idea, and I definitely intend to write up how to get things
working on the SheevaPlug.  But first, I want to get things working :)

        brian.

> W dniu 09.03.2011 06:30, Brian Gerkey pisze:
>> hi,
>>
>> I'm trying to get the Kinect working on an embedded computer with a
>> 1.2GHZ ARM.  I've gotten over the various compilation hurdles (various
>> patches filed already, a tutorial to come later), and am now to the
>> point of trying to make the device actually work.  I'm using
>> Diamondback plus the latest openni code from hg @ kforge, all compiled
>> from source.  The kinect is the only USB device in use.
>>
>> I'm seeing the following behavior: for a brief, variable period (5-45
>> seconds) after I've roslaunched `openni_camera/openni_node.launch`, I
>> can get data from the kinect.  I've seen live, valid point clouds and
>> images (both mono and color) in rviz, which is heartening.  However,
>> the dataflow always stops at some point.  To get it going again, I
>> have to kill and relaunch the driver.
>>
>> When data's flowing, I see `openni_node` and `XnSensorServer`
>> consuming significant CPU.  After data stops, those processes, along
>> with the rest of the machine, are idle.  The `openni_node` is
>> responsive to `rosnode`, roswtf identifies no problems, and I'm able
>> to subscribe to all the topics offered by `openni_node`; I just don't
>> receive any messages.
>>
>> Any thoughts?  Any debugging suggestions?  The `openni_node` is
>> surprisingly quiet; is there some way to put it in verbose / debug
>> mode or get diagnostics from it?
>>
>>       brian.
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Brian Gerkey
In reply to this post by Patrick Bouffard-2
On Tue, Mar 8, 2011 at 10:38 PM, Patrick Bouffard
<[hidden email]> wrote:
> ROS drivers aside, it would be a good sanity check to see if you can stream
> the data using freenect (maybe with one of the demo programs from that
> project). I'm not sure about apples-to-apples performance comparisons--in
> the end I was able to do what I wanted with both the freenect-based driver
> and the openni-based one. But I think there's a case to be made for the
> freenect driver's simplicity--you can easily trace back to the libusb calls.

Sounds like a good idea; I'll try freenect and report back.

        brian.

> On Tue, Mar 8, 2011 at 10:21 PM, Radu Bogdan Rusu <[hidden email]>
> wrote:
>>
>> On 03/08/2011 10:05 PM, Bill Morris wrote:
>> > Have you tried the older driver libfreenect based driver? I think it
>> > might be slightly faster.
>>
>> Bill,
>>
>> Have you made any comparisons? We'd be interested to see the results. We
>> cannot improve something unless we know it's
>> broken. :)
>>
>> Cheers,
>> Radu.
>> --
>> http://pointclouds.org
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
>
>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
>
>
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

futureignobel
In reply to this post by Brian Gerkey
Hi Brian,

> On Wed, Mar 9, 2011 at 12:05 AM, futureignobel<[hidden email]>  wrote:
>> 1. What is the frequency of published signals when you use rostopic hz
>> /camera/... ?
> There's a lot of variation in the initial rate, but it always goes to
> zero.  Here's a representative trace:
>
> $ rostopic hz /camera/depth/points
> subscribed to [/camera/depth/points]
> average rate: 6.452
>          min: 0.092s max: 0.428s std dev: 0.12204s window: 7
> no new messages
> no new messages
>
> I get a similar result for any /camera/* topic.  And after one topic
> stops, all topics stop.  At that point, openni_node and XnSensorServer
> are idle (but still responsive to ROS XMLRPC, including topic
> subscriptions).
>
>> 2. Have you used dynamic_reconfigure to minimize processors load?
> I played with it a bit, and also tried changing the params in the
> launch file.  I never produced a significantly different result.  What
> parameter setting would minimize processor load?
>
I set image resolution to QQVGA@30Hz and XYZ_unregistered and achieved
about 25Hz for rostopic hz /camera/depth/points. When the difference
between 30Hz and the obtained topic frequency was lower openni_camera
worked longest (as if this difference, here around 5Hz, decided about
the amount of data being buffered, thus the buffer-theory).
Unfortunately I cannot provide any more specific info than this what
I've remembered as I'm struggling with openni_camera myself at the
moment and cannot run it neither on BB-xM nor on my PC
(http://answers.ros.org/question/326/openni_cameraopenni_nodelaunch-wont-start).

>> 3. How much RAM does your machine have?
> 512MB
>
>> 4. Could you please, out of curiosity, tell us what board exactly are
>> you using?
> SheevaPlug: http://en.wikipedia.org/wiki/SheevaPlug
>
>> I've had similar experiences with PCL on ARM (with limited resources:
>> 1GHz, 500MB RAM). To me it seemed like the time
>> openni_camera.openni_node.launch worked seamlessly depended strongly on
>> the amount of data published to topics. So it felt like when some
>> "buffer" was overflown openni_camera stopped publishing. When I chose
>> QQVGA and used XYZ_unregistered I obtained best results (25Hz frequency
>> for pointcloud2 topic) which caused the system to work best (I achieved
>> longest times before openni_camera needed rebooting).
> That sounds exactly like what I'm seeing.  It feels like openni_node
> is somehow stateful; there's a qualitative change between it works and
> when it doesn't.  I like your hypothesis about a buffer filling up,
> but don't have any direct evidence of it.
>
> It would help a lot to get some visibility into what the driver is
> doing (talking to the device, decoding the data, etc.).  Any hints on
> getting verbose / debug output?
>
I've just started playing with ROS about a month ago so I haven't
mastered debugging yet.
>> 5. Is it possible to configure openni_camera to just publish to one
>> topic? If so, try it out.
> Well, when I'm only subscribing to one topic, I would expect the node
> to only be doing the work necessary to produce the data for that
> topic.
>
Ok, I wasn't aware of that. So nodes are only really publishing if there
is someone listening? Sounds logical, but I didn't know.

>> I've tried to install (and actually once succeeded) openni_camera on a
>> BeagleBoard-xM. I tried to summarize my know-how in a post here
>> http://answers.ros.org/question/130/openni-compilation-error-diamondback-version
>> . For now, I quit those efforts and am working on algorithms, but would
>> like to come back to it as soon as I have them working on my PC, so I
>> think it would be great to find a place (Wiki?) where we could write our
>> experiences down for future reuse.
> Good idea, and I definitely intend to write up how to get things
> working on the SheevaPlug.  But first, I want to get things working :)
>
> brian.
I unfortunately wasn't able to obtain repeatable build on BB-xM, but
only (I guess) because of PCL being resource hungry while compiling. So
cross-compilation should help, but I didn't yet manage to do it. I'd
certainly like to document everything once I successfully cross-compiled
ROS (PCL + openni_camera) on BB-xM. Tom.

>> W dniu 09.03.2011 06:30, Brian Gerkey pisze:
>>> hi,
>>>
>>> I'm trying to get the Kinect working on an embedded computer with a
>>> 1.2GHZ ARM.  I've gotten over the various compilation hurdles (various
>>> patches filed already, a tutorial to come later), and am now to the
>>> point of trying to make the device actually work.  I'm using
>>> Diamondback plus the latest openni code from hg @ kforge, all compiled
>>> from source.  The kinect is the only USB device in use.
>>>
>>> I'm seeing the following behavior: for a brief, variable period (5-45
>>> seconds) after I've roslaunched `openni_camera/openni_node.launch`, I
>>> can get data from the kinect.  I've seen live, valid point clouds and
>>> images (both mono and color) in rviz, which is heartening.  However,
>>> the dataflow always stops at some point.  To get it going again, I
>>> have to kill and relaunch the driver.
>>>
>>> When data's flowing, I see `openni_node` and `XnSensorServer`
>>> consuming significant CPU.  After data stops, those processes, along
>>> with the rest of the machine, are idle.  The `openni_node` is
>>> responsive to `rosnode`, roswtf identifies no problems, and I'm able
>>> to subscribe to all the topics offered by `openni_node`; I just don't
>>> receive any messages.
>>>
>>> Any thoughts?  Any debugging suggestions?  The `openni_node` is
>>> surprisingly quiet; is there some way to put it in verbose / debug
>>> mode or get diagnostics from it?
>>>
>>>        brian.
>>> _______________________________________________
>>> Ros-kinect mailing list
>>> [hidden email]
>>> https://code.ros.org/mailman/listinfo/ros-kinect
>> _______________________________________________
>> Ros-kinect mailing list
>> [hidden email]
>> https://code.ros.org/mailman/listinfo/ros-kinect
>>
> _______________________________________________
> Ros-kinect mailing list
> [hidden email]
> https://code.ros.org/mailman/listinfo/ros-kinect
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Brian Gerkey
On Wed, Mar 9, 2011 at 8:24 AM, futureignobel <[hidden email]> wrote:
> I unfortunately wasn't able to obtain repeatable build on BB-xM, but
> only (I guess) because of PCL being resource hungry while compiling. So
> cross-compilation should help, but I didn't yet manage to do it. I'd
> certainly like to document everything once I successfully cross-compiled
> ROS (PCL + openni_camera) on BB-xM. Tom.

A suggestion on PCL: you don't actually have to build it to use
openni_camera.  On my system, I'm stuck with an older compiler (GCC
4.3.3) that chokes on Eigen, so I can't build anything that uses Eigen
(no, Radu, I haven't yet sent the Eigen devs a bug report; will do).
But I noticed that openni_camera needs only data structures from
pcl/pcl_ros.  So I did the following:
* hack pcl/manifest.xml and pcl_ros/manifest.xml to remove all -lfoo
args from the exported lflags
* touch pcl/ROS_NOBUILD and pcl_ros/ROS_NOBUILD
Then `rosmake openni_camera` works fine, skipping the build of pcl and
pcl_ros. This clearly isn't a good long-term solution, but it got me
compiled and running.

        brian.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

futureignobel

> On Wed, Mar 9, 2011 at 8:24 AM, futureignobel<[hidden email]>  wrote:
>> I unfortunately wasn't able to obtain repeatable build on BB-xM, but
>> only (I guess) because of PCL being resource hungry while compiling. So
>> cross-compilation should help, but I didn't yet manage to do it. I'd
>> certainly like to document everything once I successfully cross-compiled
>> ROS (PCL + openni_camera) on BB-xM. Tom.
> A suggestion on PCL: you don't actually have to build it to use
> openni_camera.  On my system, I'm stuck with an older compiler (GCC
> 4.3.3)
And why is it you're stuck with GCC 4.3.3? If because of
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42748 GCC 4.5.0 solves it.
There are GCC 4.5.0 armel debian packages on Maverick, on older versions
of Ubuntu they are available on PPA's. Or are you not running Ubuntu?

>   that chokes on Eigen, so I can't build anything that uses Eigen
> (no, Radu, I haven't yet sent the Eigen devs a bug report; will do).
> But I noticed that openni_camera needs only data structures from
> pcl/pcl_ros.  So I did the following:
> * hack pcl/manifest.xml and pcl_ros/manifest.xml to remove all -lfoo
> args from the exported lflags
> * touch pcl/ROS_NOBUILD and pcl_ros/ROS_NOBUILD
> Then `rosmake openni_camera` works fine, skipping the build of pcl and
> pcl_ros. This clearly isn't a good long-term solution, but it got me
> compiled and running.
>
> brian.
Thanks, I'll try it out.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Brian Gerkey
On Wed, Mar 9, 2011 at 9:01 AM, futureignobel <[hidden email]> wrote:
> And why is it you're stuck with GCC 4.3.3? If because of
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42748 GCC 4.5.0 solves it.
> There are GCC 4.5.0 armel debian packages on Maverick, on older versions
> of Ubuntu they are available on PPA's. Or are you not running Ubuntu?

The SheevaPlug ships with Jaunty, which has GCC 4.3.3.  As I
understand it, something about the way that later Ubuntu binaries are
built makes them not work on the Marvell processor, so it's not easy
(perhaps impossible) to upgrade to Maverick.

Of course, I could build a newer GCC from source, or even install a
totally different OS (a fair number of people seem to run Debian on
these things), but I'm looking for a least-resistance (and hence more
easily repeatable, solution).   I now realize that I haven't checked
whether a newer GCC is available for Jaunty as a backport, which could
work.  In any case, I was able to work around the compiler issue by
avoiding Eigen, and so far that's been the only Jaunty/GCC-specific
problem I've had.

        brian.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Vijay Pradeep
Brian,

Here is a single data point that may or may not be useful:

I was trying to get a kinect up and running on a very powerful i7 machine.  However, I was getting performance very similar to you: data coming in at a couple hz, and then deteriorated to no data coming it.  There were actually two things causing this problem:

1) Due to an inline ethernet-usb converter, The kinect was never getting to Full Speed USB mode.  You can check this in dmesg by seeing if you get some USB error when you plug in the kinect.  Maybe your embedded board doesn't support Full Speed USB.

2) Lossy USB extension.  We generally got better results when we plugged the kinect directly into the machine's USB port.

Vijay

On Wed, Mar 9, 2011 at 10:18 AM, Brian Gerkey <[hidden email]> wrote:
On Wed, Mar 9, 2011 at 9:01 AM, futureignobel <[hidden email]> wrote:
> And why is it you're stuck with GCC 4.3.3? If because of
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42748 GCC 4.5.0 solves it.
> There are GCC 4.5.0 armel debian packages on Maverick, on older versions
> of Ubuntu they are available on PPA's. Or are you not running Ubuntu?

The SheevaPlug ships with Jaunty, which has GCC 4.3.3.  As I
understand it, something about the way that later Ubuntu binaries are
built makes them not work on the Marvell processor, so it's not easy
(perhaps impossible) to upgrade to Maverick.

Of course, I could build a newer GCC from source, or even install a
totally different OS (a fair number of people seem to run Debian on
these things), but I'm looking for a least-resistance (and hence more
easily repeatable, solution).   I now realize that I haven't checked
whether a newer GCC is available for Jaunty as a backport, which could
work.  In any case, I was able to work around the compiler issue by
avoiding Eigen, and so far that's been the only Jaunty/GCC-specific
problem I've had.

       brian.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect



--
Vijay Pradeep
Systems Engineer
Willow Garage, Inc.
[hidden email][hidden email]


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Brian Gerkey
On Fri, Mar 11, 2011 at 12:23 AM, Vijay Pradeep
<[hidden email]> wrote:
> 1) Due to an inline ethernet-usb converter, The kinect was never getting to
> Full Speed USB mode.  You can check this in dmesg by seeing if you get some
> USB error when you plug in the kinect.  Maybe your embedded board doesn't
> support Full Speed USB.

Here's what dmesg says on plugging the Kinect in:

usb 1-1: new high speed USB device using ehci_marvell and address 25
usb 1-1: configuration #1 chosen from 1 choice
hub 1-1:1.0: USB hub found
hub 1-1:1.0: 3 ports detected
usb 1-1.2: new full speed USB device using ehci_marvell and address 26
usb 1-1.2: configuration #1 chosen from 1 choice
usb 1-1.1: new high speed USB device using ehci_marvell and address 27
usb 1-1.1: configuration #1 chosen from 1 choice
usb 1-1.3: new high speed USB device using ehci_marvell and address 28
usb 1-1.3: configuration #1 chosen from 1 choice

That looks ok to me, but then I'm not sure what it's supposed to look like...

> 2) Lossy USB extension.  We generally got better results when we plugged the
> kinect directly into the machine's USB port.

I have the Kinect plugged direclty into the one USB port on the box.
I haven't cracked the box open to see what happens on the other side
of the connector, but I can only hope that there's nothing silly going
on in there.

        brian.
_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Vijay Pradeep

Here's what dmesg says on plugging the Kinect in:

usb 1-1: new high speed USB device using ehci_marvell and address 25
usb 1-1: configuration #1 chosen from 1 choice
hub 1-1:1.0: USB hub found
hub 1-1:1.0: 3 ports detected
usb 1-1.2: new full speed USB device using ehci_marvell and address 26
usb 1-1.2: configuration #1 chosen from 1 choice
usb 1-1.1: new high speed USB device using ehci_marvell and address 27
usb 1-1.1: configuration #1 chosen from 1 choice
usb 1-1.3: new high speed USB device using ehci_marvell and address 28
usb 1-1.3: configuration #1 chosen from 1 choice

That looks ok to me, but then I'm not sure what it's supposed to look like...

Looks good... I don't see errors.  Unfortunately I can't remember exactly what the error was, but it looked something like:
USB 2.0 device failed to reach full speed mode. dropping down to [something] mode.

Looks like you're not having the same problems as me then.

Vijay

--
Vijay Pradeep
Systems Engineer
Willow Garage, Inc.
[hidden email][hidden email]


_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

Brian Gerkey
In reply to this post by Brian Gerkey
On Wed, Mar 9, 2011 at 7:52 AM, Brian Gerkey <[hidden email]> wrote:

> On Tue, Mar 8, 2011 at 10:38 PM, Patrick Bouffard
> <[hidden email]> wrote:
>> ROS drivers aside, it would be a good sanity check to see if you can stream
>> the data using freenect (maybe with one of the demo programs from that
>> project). I'm not sure about apples-to-apples performance comparisons--in
>> the end I was able to do what I wanted with both the freenect-based driver
>> and the openni-based one. But I think there's a case to be made for the
>> freenect driver's simplicity--you can easily trace back to the libusb calls.
>
> Sounds like a good idea; I'll try freenect and report back.
Ok, I tried freenect as well.  I'll summarize my experience here, in
case it's useful to others trying to use the Kinect on a small
computer.

        brian.

= The goal =
Combine a Kinect with a SheevaPlug
(http://en.wikipedia.org/wiki/SheevaPlug) to create a small,
inexpensive way of deploying RGB-D sensors in a building.  For US$100,
the SheevaPlug has a 1.2GHz Marvell ARM, 512MB RAM, ethernet, USB
master and slave, and SDHC slot.

= Quick summary =
Neither openni_node nor kinect_node worked.  I was able to get partial
functionality from my own libfreenect-based node.  The code is
attached.

= The details =
In every configuration I tried, with all nodes, I saw the following behavior:
(Phase 1) The driver would work ok for some time, with data being
successfully being delivered.  During this time, the driver was quite
busy, sometimes consuming the entire CPU.  There were around 1200
interrupts per second (according to vmstat).
(Phase 2) At some point, the driver would stop working, leading to no
data delivery.  During this time, the driver was completely idle, and
there were around 100 interrupts per second, which is the steady state
for this computer at idle.  I never saw a transition out of this
phase.

The thing that varies between configurations is how long we remain in
Phase 1.  Data points:
* openni_node remains in Phase 1 for anywhere from a few to about 30 seconds
* kinect_node remains in Phase 1 for 0 seconds (never delivers data)
* my node (code attached) remains in Phase 1 indefinitely (haven't yet
seen it stop working)

== My simple node ==
After failing to get good behavior from openni_node and kinect_node, I
set out to write the simplest possible node that could ship Kinect
data off the SheevaPlug to the network.  The result is attached.
Pardon the style; it began life as a pure C libfreenect application.

This simple node only ever asks for one of rgb or depth (never both),
and ships the incoming data out with minimal computation.  Examples:
./simple _mode:=rgb
./simple _mode:=depth
You can tell it how many frames to skip publishing, e.g. to publish
every 5th depth image:
./simple _mode:=depth _skip:=4

If you modify the code to ask for rgb and depth simultaneously, it
goes to Phase 2 after about 60 seconds.

Notes:
* The MONO8 encoding of the depth data is wrong, but was a quick way
to produce something that can be visualized (e.g., in image_view)
* The Kinect has to be unplugged and replugged between each use;
otherwise data streaming doesn't start.  I don't know the cause.

= Conclusions =
My hypothesis (and I'm not sure that this makes sense) is that the
transition to Phase 2 happens when the driver loses sync with the
firehose of USB data from the device.  After that point, all is lost,
with no chance of recovery.  This transition can be hastened by
either: (i) asking for more data from the device (e.g., rgb + depth at
the same time), or (ii) taking longer to service the USB stream (e.g.,
doing computation between reads).

It seems that the SheevaPlug is right at the edge of what's needed to
get data from the Kinect.  If you ask for too much, or try to do too
much work, it'll stop working.  I don't know which resource is being
exhausted.  It could be CPU generally, or kernel cycles to pull data
from USB, or the USB driver, or the USB hardware, or something else.

To achieve the goal of a solid infrastructural deployment of RGB-D
sensors, I recommend trying a different, more powerful computer.

_______________________________________________
Ros-kinect mailing list
[hidden email]
https://code.ros.org/mailman/listinfo/ros-kinect

simple.cpp (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Ros-kinect] kinect on ARM

jako99
In reply to this post by Brian Gerkey
Hi there,

interesting.

I tried to make the Kinect work with the Nokia n900 but I got problems too.

I tried both the drivers (libfreenect and primesense). I had to write a patch for the kernel in order to make the kinect start but the results are not so good.

I modified the libfreenect in order to get the data even in the case of no complete frames.
The problem is that I get a lot of packet loss and packet corruption (bad magic numbers).

After each start of frame, usually the first packets are ok (and so the first raws of the image are ok) but after packets start to get corrupted (also the sequence number) and there is a packet loss too.
The strangest thing is not this. It's the fact that the sequence number between two SOF(start of frame) is almost never correct.
For example, there are 162 packets for one frame for the color camera. If the seq number of the frame 1 is 0, the seq number of the second SOF frame packet is not 163 but is a different number. The EOF (end of frame) is sometimes (correctly) before the SOF but often it is in another position.
I checked also the timestamps but there is really no way to get a single frame :(

I overclocked also the cpu up to 1.15 Ghz but no improvements :(

Loading...