Counting Bees on a Raspberry Pi with a ConvNet | Hacker News Reader

Counting Bees on a Raspberry Pi with a ConvNet

matpalm.com

177 pointsburningion23 comments

Awesome! Make a nice enclosure and you have a niche hardware product.

Once you get the JeVois[1] it would be interesting to run the "Background subtraction to detect moving objects"[2] model and see how that compares.

1. https://www.jevoisinc.com/products/jevois-a33-smart-machine-...

2. http://jevois.org/moddoc/DemoBackgroundSubtract/modinfo.html

the protect is very cool. As a professional deep learning researcher myself, the author couldn't made a few things to make his life more simple, and for better results. For example, using "Image translation" (encoder decoder structure) is what is used everywhere in the regular semantic segmentation models, of which there are many pretrained and already available for low computing power mobile, and which would have saved the author a lot of time by bit having to hand label images, and would likely have done a better job because they were trained on gigantic datasets instead of relatively few hand labeled images.

The normal way to do this would be to use mobile net v2 with the tensorflow image detection pipeline:

https://github.com/tensorflow/models/blob/master/research/ob...

Awesome job!! It look pretty similar to, what the german startup apic.ai is doing :) Here is the google translator link: https://translate.google.com/translate?hl=en&sl=auto&tl=en&u...

Does anyone have an idea roughly how long this would take to build?

I'm new to CS and I'm always seeing all these cool projects I want to work on, but I don't know how much of a time commitment it really is to build a side project like this.

It is strongly a factor of the skill of the builder.

Juniors will probably get tripped up getting the environment setup

Medium level will probably get a simple one working OK but be unable to tune it (e.g. get stuck in execution speed or a false positive problem)

Senior engineers might be able to do it in 1-2 focused days.

But regardless of your skill level you should just attempt it. It doesn’t matter if you don’t get all the way through, you will learn loads in the process, and if you get stuck your subconscious will wittle away and you’ll have some lighting bolt moment while you’re in the shower, or biking, or playing games. If you’re new to the field go for it! If you get stuck and need some direction, feel free to email me

Say you'd train this on images of bees of some size. Would the same net also work for bees which appear to be (e.g.) twice as large?

Maybe, but you shouldn't count on it. Perhaps a big bee is supposed to be counted as two small bees, you didn't specify when you trained it.

no, it won't directly. conv nets handle translation invariance but not scale invariance. having said that there's no reason you can't use aggressive data augmentation for this (resizing before patch sampling). i wonder how much the semi supervised approach might help too; if you've labelled _only_ small bees in a subset of the data, trained a model, applied to a larger dataset & retrained there will be a small amount of detections (that are true positives) to bees that are slightly larger (and smaller) than the ones you labelled.... (maybe?)

Ok. Why isn't there a kind of network function that works with scaling, like convolution works with translation?

I think a CNN can handle scale invariance e.g. With max pooling

>the first baseline was to freeze the tensorflow graph and just run it directly on the pi. this works without any problem, it's just the pi can only do 1 image / second :/

I wonder if this was done using full blown tensorflow, which might be leaving a lot of performance on the table. There are currently some efforts to improve the efficiency/optimization of getting a network to run well on the modest hardware of a pi e.g., tensorflow lite: https://www.tensorflow.org/mobile/tflite/ tvm: tvm.ai

That's just a sanity check; the end goal is some accelerated hardware ( neural ompute stick / jevois / etc)

"my first quick experiment was a patch based "bee / no bee in image" detector. "

What are these patches and what was your intuition on starting w/ it?

Just a randomly sampled 32x32 patch, intution being most bees in image aren't bigger

fully convolutional neural networks: https://www.quora.com/How-is-Fully-Convolutional-Network-FCN...

This is extremely cool - great work!

Awesome project & blog!

But what does it count now, exactly? Only the number of bees in the air in some space before the hive, right? It doesn't even say if they're going in or out? What exactly can be measured with this metric, apart from a very generic 'activity level'? Can it detect swarming, robbing, total bee count of a hive, bearding, anything useful?

Looks like the whole thing is up on GitHub so you're free to do what you want with it.

Honestly, your comment comes off as complaining that the in-flight wifi is too slow. He found an interesting problem and did a great job documenting his process + approach, if that doesn't belong in HN I don't know what does.

As someone who grew up with bee hives, while "activity level" is generic, it's also very interesting at the same time. Spikes in "activity level" (like the chart shows at 4pm) would be very useful to me, meaning "Something abnormal is going on".

>It doesn't even say if they're going in or out? What exactly can be measured with this metric, apart from a very generic 'activity level'? Can it detect swarming, robbing, total bee count of a hive, bearding, anything useful?

AFAIK it can't do this yet, but detecting bees in an image of the entrance is a first step towards doing any more advanced monitoring. A good next step is to track bees across successive frames, which gets you a) approximately how many bees enter/leave the camera view at each timestep and b) approximately how many bees enter/leave the hive at each timestamp. From that you can integrate to get the total number of bees moving in and out each day (which isn't the population, but is presumably correlated).

yeah, that's true it's only counting bees in one image, which i thought was part 1 of any of these other things. can you post your approach / code to doing any of these things you mention? i'd love to see what you've done so far.

Reply on news.ycombinator.com