Replicating PixyMon w/ libpixyusb2

crazyquark · November 18, 2018, 12:40pm

Hello,

I am trying to figure out how PixyMon is pulling both the raw frame and result of the running program without suspending the program.
AFAIK, this is not possible via libpixyusb2 so it must use the raw chip protocol.
What I am trying to achieve is to stream the video while also showing detected objects on top - like PixyMon but from my own program that uses libpixyusb2.

All I found was:
stop program
get raw frame
resume program

But I suspect that will impact negatively FPS.

Can you help me?
Thank you

BTW, I’ve been experimenting with modifying the Python wrapper to expose the get raw frame command:
See this:

github.com

crazyquark/pixy2/blob/master/src/host/libpixyusb2_examples/python_demos/get_raw_frame_python_demo.py

from ctypes import Structure, c_uint

import pixy
from pixy import *

pixy.init()
pixy.change_prog('color_connected_components')

frame_width = 316
frame_height = 208
raw_frame = Uint32Array(frame_width * frame_height)


class Blocks (Structure):
    _fields_ = [("m_signature", c_uint),
                ("m_x", c_uint),
                ("m_y", c_uint),
                ("m_width", c_uint),
                ("m_height", c_uint),
                ("m_angle", c_uint),

This file has been truncated. show original

You will need to rebuild _pixy.so using my fork of the source code.

edge · November 20, 2018, 12:11am

Hello crazyquark,
Pixymon uses a special streaming mode to get the video data and object data. It’s efficient in that it doesn’t interrupt the program running on Pixy. But it still is only about 20 frames per second over USB vs Pixy’s 60 frames per second native framerate. We haven’t exposed this type of streaming via libpixyusb2 (sorry!)

Edward