<< return to Pixycam.com

Getting too many blocks with get_blocks function from python

Pixy Software
#1

When I call get_blocks with a single object, I get a large number of blocks returned although other apps (pixymon and hello_pixy) only get a single block. I’m assuming there is something wrong on my side, but was hoping this might sound familiar as to give me a pointer as to where to investigate.

If I start out without an object I will correctly get 0 blocks. Then if I have a single object I will get max_blocks (25 for example) returned. In fact if I remove the object, I will continue to get 25 blocks for about 8-10 calls and then I will get 0 blocks again. It is as if there is a list of stale blocks stored in the firmware, and all of them are continuing to be sent until the buffer is flushed. I’ve looked at the blocks and they seem to be slight variations of the correct block.

The only thing I’m doing differently is that this is my python3 code. I don’t have this problem with your swig example (I get 1 to 8 blocks max), but as I’m not familiar with swig, I’d prefer not to switch over. All of the rest of my code is working.

I have this issue with the 1.02 and the 2.0.2 firmware.

I’ve seen some similar issues posted. I’ll continue debugging, but any suggestions would be appreciated.

#2

So this has nothing to do with python. I have reproduced this with a modified version of “hello_pixy”. The issue is that I am calling get_blocks intermittently in my application. It only seems to work properly if it is in a tight loop. I added a 2 second delay every 15 times a block is detected and I always get a large number (25) of blocks detected after the delay.

I’ve included my test app. I run it with the object (I’m using a blue ball) in front of the camera in the same position for the entire run of test.

Let me know if you can reproduce this and if you know what is going on. Note that I’m using firmware 2.0.2 right now.

#3

Definitely it is not about your code. I am getting something similar. I call the get_Blocks() function in an infinite loop for debugging while showing in PixyMon 5 blocks from 2 different signatures and what I get is an 1 in the beggining, then a 2. If I Change the blocks or signatures, I am still getting 2, sometimes 1 or sometimes 0. I cannot find a way how to know when Pixy is not detecting anything, because sometimes I also get a 0 when there is something in PixyMon (and I am using the Default program, no raw Video or cooked Video)

#4

I’ve confirmed that if I have a background task that continuously get’s the blocks (and ignores them), then my intermittent get_block works as expected. Although this is a workaround, it is not very efficient. Is it possible for you to implement a flush command on your side. This would flush whatever the buffer is that you are collecting the blocks in. I can then flush first then call get_blocks. (I’d try and do it myself but I don’t have Keil tools).

#5

Shari,

In libpixyusb, a thread is spawned which reads messages from the USB connected to Pixy. When a message containing a block comes in, the thread stores the block object in a container. Currently, there is no way to “flush” the vector container except by calling pixy_get_blocks().

I’m assuming the behavior you desire for pixy_get_blocks() is for the function to return blocks that represent the current state of what Pixy is seeing, and not all the blocks that have ever been received from Pixy since the last pixy_get_blocks() call.

If we wanted pixy_get_blocks() to only return blocks that represent the current state of what Pixy is seeing at this moment we will have to implement a correlation algorithm in the libpixyusb read thread.

You could experiment with this by modifying the functions: @PixyInterpreter::add_normal_blocks()@ and @PixyInterpreter::add_color_code_blocks()@ in the file @pixy/src/host/libpixyusb/src/pixyinterpreter.cpp@ which are part of the libpixyusb source code. No Keil compiler is required as this is on the host side.

Thanks,
John

#6

I created this function which works for me. I don’t think it is foolproof based on the timing of when I call this and getblocks, but so far so good.

void PixyInterpreter::flush_blocks()
{
uint32_t index;
Block block;

// Wait for permission to use blocks_ vector //
blocks_access_mutex_.lock();

for (index = 0; index != blocks_.size(); ++index) {
    blocks_.erase(blocks_.begin());
}
blocks_access_mutex_.unlock();

}

#7

Shari,

I’ve added a new function to libpixyusb called pixy_blocks_are_new() that can be called to get the status of the blocks received from Pixy that should be exactly what you’re looking for. The hello_pixy example has been updated to use this as well.

This is available with the latest commit: cad657532271b8b59cb9e8c21e4016612f032d45

Thanks,
John

#8

So the change is that you know don’t remove any blocks when get_blocks is called, instead you have a flag to say whether you have previously retrieved the blocks?

I think the major improvement for me is the addition of
blocks_.clear();
to
PixyInterpreter::add_normal_blocks

This was equivalent to my flush command. I think that is all I need. I’m not quite sure about the blocks_are_new call. It seems like it is only useful if you haven’t added a new block, because once you do you will clear all blocks again.

Thanks!

#9

So Pixy creates brand new complete set of data every 20 ms for every frame. And when new data comes in, the old data is now completely worthless. Pixy is a sensor…

As a Pixy user, you just want to ask Pixy — hey, what’s your best guess about what’s out there? And of course, the best guess (in Pixy’s case) are the blocks from the most recent frame.

John recently (yesterday) modified libpixyusb so it does just that – each time you call get_blocks() you are getting the most recent data, the most recent blocks. Before the modification libpixyusb was queuing all of the blocks and returning them when you called get_blocks() (so if a couple frames elapsed since the last time you called get_blocks, you’d get several frames of blocks — or if no frames elapsed since the last time you called get_blocks, you’d get no data) So… this is more intuitive.

pixy_blocks_are_new() is a bonus function that tells you if a new frame has come in since the last time you called get_blocks().

#10

My desire for pixy_get_blocks() is for the function to return blocks that represent the current state of what Pixy is seeing, flush all blocks and then only return new blocks and ignore the previous existing blocks since the last pixy_get_blocks() call.

Example…

  1. 2 purple dinosaurs detected and stay in the arena. Blocks detected return 2.
  2. 1 more purple dinosaur enters the arena. Blocks detected return 1.

How should I approach this? Can I use a flush command?

#11

Hello David,
Pixy does not remember which objects appear or disappear between frames, so the kind of functionality you describe isn’t supported right now. You might consider writing a piece of code that determines correspondence of the detected blocks between frames. This code essentially uses the “raw” blocks that Pixy reports and then only outputs new objects, for example.

We’ve played a little with this in the past because it’s useful, but it’s challenging to solve the correspondence problem (finding the matching blocks between frames). This is fairly common problem in computer vision, and it’s solvable by making some assumptions.

Sorry, that may be more information than you wanted, but hope it helps! Let me know if you have any further questions. I’m logging this as a feature request.

Edward

#12

dont call pixy more than every 21 milliseconds. If you do, you might not get any blocks even if there are blocks. Just add a delay to your code or check the time passed before each call.