Tutorial standard configuration not giving good place cells

[dlevenstein]

Suggests a change during implementation that resulted in hyperparameter or computation mismatch from original paper...

[dlevenstein]

@acumpelik noticed as well, so feel free to join the diagnostic team 😉

[dlevenstein]

Some places to start
-use CellTypeAnalysis to quantify single field place cells (instead of by eye)
-confirm hyperparameters match what's in the paper
-run hyperparameter sweeps to compare to paper
-try thRNN_5win (old implementation) to see if it matches masked k=5

[dlevenstein]

After looking back at my old trained networks, I think it may be more similar than we thought (see e.g. Figure S10, S16b, and S17 from the revised manuscript, attached). Here's my proposed plan of attack for closing this issue:

1. Run CellTypeAnalysis to confirm quantitative similarity in % single-field place cells.

2. Match hyperparams from paper perfectly (One hyper parameter difference I've found so far: --bias_lr=0.1)

3. Modify the tutorial/docs

• End the basic analysis tutorial/documentation with CellTypeAnalysis to quantify number of each cell type
• Point out that there are not many place cells and next we'll learn about some architecture changes that increase number of place cells - Link to Models documentation as the next step
• Add sparse-lognormal architecture of the models documentation (= issue Documentation: different pRNN architectures #21)

4. Use rollout/sparse-lognormal architecture as our default architecture going forward

[dlevenstein]

TL;DR from @AHalawa99 's investigations.... looks like this issue is due to a change in the way bias_lr is handled. in earlier versions of the code it was matched to recurrent weights LR. (i.e. scaled by 1/root(fan_in)). At some point I updated it to just be separately scaled to the global LR. However bias_lr=0.1 is much higher than the recurrent lr...

running a hyper parameter sweep now of N vs bias_lr to determine the best scaling for consistent behavior across different N and will update the codebase/defaults to match.

@acumpelik you might want to run a network with bias_lr=0.01 to see if it gives you nicer fields, or just wait for the scaling results

[dlevenstein]

good news is that adel validated that everything looks great with the refactoring - gj @vviggyy !