Myujth

There are 2 cameras Cam1 & Cam2 mounted in a production line. They are provided with sensors those signal my program once a product touches one of them. Those cameras are placed one behind the other.

After Cam2, a balance B is installed and provided with a sensor as well. When the product arrives there, the product will be weighed and signal my program with the weight. Then my program must do the evaluation for this product based on the two captured images and data received from B (weight).

Actually, one/two products can fit between Cam1 & Cam2. and also one/two products can fit between Cam2 & B.

I faced some difficulty to find a stable algorithm to figure out if the images, as well as the weight, belong to the same product. There are some hard conditions cannot be changed inside the factory:

1. Undefined speed for the line. (the line can be stopped for any
   reason at any time, it can accelerate/speed down, it can be moved manually by the workers there, ...)


2. a product can be removed after Cam1:
   
   
   • between Cam1 & Cam2 ==> only captured by Cam1 BUT no image from Cam2 and no data from B
   
   
   • between Cam2 & B ==>
     captured by Cam2 & Cam1 BUT no data from B
   
   


3. no way to stick such kind
   of readable chip with unique number so we can read on Cam2 & B to
   make sure for synchronicity.

This problem must be solved programmatically. The factory has fully obstacles that avoid me to make any modification there.

My initial solution is to assign a FIFO for each camera, FIFO_Cam1 & FIFO_Cam2. once my program receives the signal from one of the cameras the convenient FIFO enqueues the captured image of the product.
Then, when I receive a signal from B, I dequeue the top image from each FIFOs and consider they belong to the same product which has the weight received with the last signal too.

Unfortunately, my consideration can only solve the problem of speed (Nr. 1) but not all cases of (Nr. 2).

I tried to make a threshold when the number of products saved into FIFO_Cam1 exceeds 5, I will signal the line with DESYNCH but doesn't work properly and cannot solve the most of cases.

Any help or idea would be appreciated!

Thanks.

You need a signal to let you know the line stopped or slowed down dramatically.

So to take care of missing products I would store a timestamp of when the picture/measurement was taken.

If the line goes at the same speed you should always see the same time difference between the events. If a product goes missing then the next measurement will come at about 2x the time required, so you can look in your fifo for an item that would match the time delta you're expecting. You can also keep track of consecutive timestamps in the same queue. It would also tell you how many products were likely skipped.

To allow for some change in the line speed you should update the time delta on an ongoing basis (last X successful items). Obviously if the line stops you need to allow for the stopped time (that's why you need a signal for it, I assume it would be easier the tagging every product). If you can measure the line speed it would be even better as you can adjust your expected deltas more accurately.

This is still somewhat approximate solution. Make sure you allow for some error in the measurements. If the line speed changes dramatically or if the number of removed products is too high this might still not work. For example, say the first camera scanned 10 products, then the line stopped for a while then starts at a different speed, somebody kicked 8 products off the line and now you get the next 2 picture. It's impossible to say for sure how to pair the later pictures to first ones.