# DenseNet **URL:** \ **Category:** Paper Reading Group\ **Created:** [September 20, 2021, 6:46pm UTC](https://community.wandb.ai/t/densenet/718 "2021-09-20T18:46:33Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![lessing](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/lessing/32/200_2.png) [@lessing](https://community.wandb.ai/u/lessing)\ **Post date:** [September 20, 2021, 6:46pm UTC](https://community.wandb.ai/t/densenet/718/1 "2021-09-20T18:46:33Z")
Register: [wandb.me/prg](http://wandb.me/prg) DenseNet Paper: [[1608.06993] Densely Connected Convolutional Networks](https://arxiv.org/abs/1608.06993) Blog post: [DenseNet Architecture Explained with PyTorch Implementation from TorchVision | Committed towards better future](https://amaarora.github.io/2020/08/02/densenets.html) Livestream on YouTube here: [W&B Paper Reading Group: DenseNet - YouTube](https://www.youtube.com/watch?v=Cbd-4ieBjMY) ---
**Author:** ![vinayak\_nayak](https://avatars.discourse-cdn.com/v4/letter/v/439d5e/32.png) [@vinayak\_nayak](https://community.wandb.ai/u/vinayak_nayak)\ **Post date:** [September 21, 2021, 5:04pm UTC](https://community.wandb.ai/t/densenet/718/2 "2021-09-21T17:04:37Z")
Yes can hear you very well Aman! ---
**Author:** ![girijesh](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/girijesh/32/39_2.png) [@girijesh](https://community.wandb.ai/u/girijesh)\ **Post date:** [September 21, 2021, 5:05pm UTC](https://community.wandb.ai/t/densenet/718/3 "2021-09-21T17:05:42Z")
one between each layer and its subsequent layer—our network has L(L+1)/2 direct connections. I couldn’t comprehend the above line from the abstract! Can we discuss it in detail? ---
**Author:** ![ravimashru](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/ravimashru/32/116_2.png) [@ravimashru](https://community.wandb.ai/u/ravimashru)\ **Post date:** [September 21, 2021, 5:12pm UTC](https://community.wandb.ai/t/densenet/718/4 "2021-09-21T17:12:08Z")
> [@girijesh](#): > > our network has L(L+1)/2 direct connections This is because the first layer has 1 connection, the second layer has 2 connections, and so on… so if we add all these connections, 1 + 2 + 3 + … + L = L(L+1)/2. ---
**Author:** ![girijesh](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/girijesh/32/39_2.png) [@girijesh](https://community.wandb.ai/u/girijesh)\ **Post date:** [September 21, 2021, 5:13pm UTC](https://community.wandb.ai/t/densenet/718/5 "2021-09-21T17:13:43Z")
yeah got it, like arithmetic series 🙂 ---
**Author:** ![girijesh](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/girijesh/32/39_2.png) [@girijesh](https://community.wandb.ai/u/girijesh)\ **Post date:** [September 21, 2021, 5:17pm UTC](https://community.wandb.ai/t/densenet/718/6 "2021-09-21T17:17:44Z")
We need to keep size intact with padding while doing convolution ? ---
**Author:** ![vinayak\_nayak](https://avatars.discourse-cdn.com/v4/letter/v/439d5e/32.png) [@vinayak\_nayak](https://community.wandb.ai/u/vinayak_nayak)\ **Post date:** [September 21, 2021, 5:17pm UTC](https://community.wandb.ai/t/densenet/718/7 "2021-09-21T17:17:55Z")
Strided (with a really large stride) `1 x 1` convolution? ---
**Author:** ![vinayak\_nayak](https://avatars.discourse-cdn.com/v4/letter/v/439d5e/32.png) [@vinayak\_nayak](https://community.wandb.ai/u/vinayak_nayak)\ **Post date:** [September 21, 2021, 5:24pm UTC](https://community.wandb.ai/t/densenet/718/8 "2021-09-21T17:24:37Z")
This means we’ll never be adding input from first block to last block or more generally, there is no skip connection across blocks but only within the blocks, right? ---
**Author:** ![durgaamma2005](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/durgaamma2005/32/117_2.png) [@durgaamma2005](https://community.wandb.ai/u/durgaamma2005)\ **Post date:** [September 21, 2021, 5:38pm UTC](https://community.wandb.ai/t/densenet/718/9 "2021-09-21T17:38:47Z")
how 32 features are getting added in db1 ---
**Author:** ![prateekagrawal](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@prateekagrawal](https://community.wandb.ai/u/prateekagrawal)\ **Post date:** [September 21, 2021, 5:39pm UTC](https://community.wandb.ai/t/densenet/718/10 "2021-09-21T17:39:42Z")
What is the reason/intuition behind using more layers in the deeper blocks (Dense Block 3 and 4).? ---
**Author:** ![girijesh](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/girijesh/32/39_2.png) [@girijesh](https://community.wandb.ai/u/girijesh)\ **Post date:** [September 21, 2021, 5:41pm UTC](https://community.wandb.ai/t/densenet/718/11 "2021-09-21T17:41:07Z")
Will this architecture will not overfit as having too many conv densely connected ? and at last one FC too ? ---
**Author:** ![ramesh](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/ramesh/32/98_2.png) [@ramesh](https://community.wandb.ai/u/ramesh)\ **Post date:** [September 21, 2021, 5:41pm UTC](https://community.wandb.ai/t/densenet/718/12 "2021-09-21T17:41:34Z")
I am confused on 1x1 with 128 filters as BottleNeck. Is that same across all the blocks and all the layers in each block? ---
**Author:** ![msampat](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@msampat](https://community.wandb.ai/u/msampat)\ **Post date:** [September 21, 2021, 5:42pm UTC](https://community.wandb.ai/t/densenet/718/13 "2021-09-21T17:42:01Z")
In the transition block, could we have only the pooling layer and avoid the 1x1 layer ? What is the advantage of the 1x1 layers : where we go from 64x56x56 to 128x56x56 ---
**Author:** ![durgaamma2005](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/durgaamma2005/32/117_2.png) [@durgaamma2005](https://community.wandb.ai/u/durgaamma2005)\ **Post date:** [September 21, 2021, 5:44pm UTC](https://community.wandb.ai/t/densenet/718/14 "2021-09-21T17:44:31Z")
fundamental doubt: how do you differentiate between filters. like each kernel is unique in a stack of kernels in filter. so when out channels depends on no of filters we apply, how exactly we differentiate between filters. since no of kernels will be equal all across filters, is kernel values are arbitrary or random values so that we can get various weight values so that each filter will be different? ---
**Author:** ![sahilcodes03](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sahilcodes03](https://community.wandb.ai/u/sahilcodes03)\ **Post date:** [September 21, 2021, 5:45pm UTC](https://community.wandb.ai/t/densenet/718/15 "2021-09-21T17:45:16Z")
Fundamental Question - When we go deeper with layers, the features that get extracted are called low level features or high level features? ---
**Author:** ![msampat](https://avatars.discourse-cdn.com/v4/letter/m/50afbb/32.png) [@msampat](https://community.wandb.ai/u/msampat)\ **Post date:** [September 21, 2021, 5:46pm UTC](https://community.wandb.ai/t/densenet/718/16 "2021-09-21T17:46:20Z")
Thanks @amanarora . that makes sense. trying it out 🙂 ---
**Author:** ![ramesh](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/ramesh/32/98_2.png) [@ramesh](https://community.wandb.ai/u/ramesh)\ **Post date:** [September 21, 2021, 5:46pm UTC](https://community.wandb.ai/t/densenet/718/17 "2021-09-21T17:46:28Z")
That makes sense. But the first layer inside the DenseBlock has only 32 outputs. So the 1x1 that follows it is not a Bottleneck anymore if it increases the feature maps to 128. But the term Bottleneck makes sense in the later layers in the DenseBlock. ---
**Author:** ![ghosh-r](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/ghosh-r/32/78_2.png) [@ghosh-r](https://community.wandb.ai/u/ghosh-r)\ **Post date:** [September 21, 2021, 5:47pm UTC](https://community.wandb.ai/t/densenet/718/18 "2021-09-21T17:47:23Z")
The ResNet architectures also add skip connections. DenseNet also adds connections from earlier layers to latter layers **inside the DenseNet Blocks**. How do you form your intuition about how this is helping the overall performance? How do this kind of _skip-connection-type_ connections improve over the performance of vanilla ResNets? ---
**Author:** ![nanthony](https://avatars.discourse-cdn.com/v4/letter/n/bbe5ce/32.png) [@nanthony](https://community.wandb.ai/u/nanthony)\ **Post date:** [September 21, 2021, 5:47pm UTC](https://community.wandb.ai/t/densenet/718/19 "2021-09-21T17:47:30Z")
Can you please quickly explain how a `1x1 conv` changes the number of channels? I thought it would be the same number… what don’t I get? ---
**Author:** ![bharatr97](https://avatars.discourse-cdn.com/v4/letter/b/0ea827/32.png) [@bharatr97](https://community.wandb.ai/u/bharatr97)\ **Post date:** [September 21, 2021, 5:48pm UTC](https://community.wandb.ai/t/densenet/718/20 "2021-09-21T17:48:22Z")
Are there any ablation studies done for the DenseNet? Like do we know which of the connections inside the dense block contribute how much to the performance? Are skip connections more valuable than direct connections? [Next page](https://community.wandb.ai/t/densenet/718.md?page=2)