merged pull#1271 for locally connected layer

[raingo]

Merge #1271 to master branch. Everything is copied from there, except some little adjustments for the master branch

@jackculpepper, @shelhamer please have a look.

[jklontz]

👍 Hope this gets merged!

[swamiviv]

Is it possible to merge this to master ?

[JordanCheney]

would love to see this merged into master!

[jackculpepper]

Thanks for bringing it up to date. Glad to know it was useful to others! I haven't been actively working on this recently but would be happy to pick it up again.

We should probably get rid of the test filler from include/caffe/filler.hpp. The test that uses it:

https://github.com/BVLC/caffe/pull/3068/files#diff-01bba204a2be65f3a1e5d5ba33b8b077R81

..was useful during debugging, but the operation performed by that filler could be moved into the test itself.

[dougsouza]

Could somebody please resolve the conflicts and merge this?

[jklontz]

We're offering a bounty to anyone willing to complete the remaining work on this pull request. See caffe-users for details: https://groups.google.com/forum/#!topic/caffe-users/N50S97Mw2VQ

[matthieudelaro]

@jklontz: #4251
Here is an attempt to finish the remaining work to merge raingo's implementation of locally connected layers into master branch:

• merge conflicts have been solved
• the filler (declared for test purposes only) has been moved inside the test itself

[kevin-li]

Would be REALLY excited to see this merged! My project needs this!

[GabrielLin]

I hope it can be merged.

[matthieudelaro]

Me too. But here is the thing: commits must be rebased (just one commit per author), and an optimization in CUDA is expected (I hope BVLC will accept it without it though, so that at least people can use it, and then anyone can improve) #4251. I hadn't time to finish this, and I won't be able to complete it before months. So... if anyone is up for rebasing, you are welcome :)

[raingo]

Closing this. Because caffe has everything needed to implement locally connected layer. Check the following layer combinations:

BxRxC = Im2colLayer(BxCxHxW)
BxD = ReshapeLayer(BxRxC)
BxD1 = InnerProductLayer(BxD)
BxC1xH1xW1 = ReshapeLayer(BxD1)

There may be mistakes, but you got the idea.

[knsong]

The idea is good! BTW, shouldn't there be a Crop layer at the very top? @raingo

[Banus]

@raingo, I wasn't able to implement the combination you proposed.
The output of Im2colLayer is a matrix (H1 x W1, Kw x Kh x C) where (Kw, Kh) is the shape of the filter kernel, and in LocalLayer each row needs to be multiplied by a different weight matrix in order to generate the (H1 x W1, C1) output.
InnerProductLayer, by acting on the flattened input, can scale each row by a different set of weights, but unfortunately it sums all the results together. You can see the issue from the different number of parameters: the weight matrix for the InnerProductLayer is (C x H1 x W1 x Kw x Kh, C1 x H1 x W1), while the LocalLayer requires only C1 x C x H1 x W1 x Kw x Kh weights (H1 and W1 appear only once).
A ConvolutionLayer also doesn't help because of the weight sharing across different input rows.

[knsong]

@Banus @raingo Here is a related question on StackOverflow.