[FEATURE] timm.models.adapt_input_conv: beyond RGB weights

Author: adamjstewartCreated Feb 25, 2025Updated Feb 26, 2025
Labelsenhancement

Is your feature request related to a problem? Please describe.

TorchGeo provides a number of model weights pre-trained on non-RGB imagery (e.g., Sentinel-2, 13 channels). Oftentimes, when dealing with time-series data, we would like to stack images along the channel dimension so that we end up with $$B \times TC \times H \times W$$ inputs. However, we don't yet have an easy way to adapt our pre-trained weights to match.

Describe the solution you'd like

timm.models.adapt_input_conv provides a powerful tool for repeating and scaling weights to adapt to changing in_chans, but only seems to support 3-channel weights if in_chans > 1. I would like to extend this to support any number of channels. Would this be as simple as replacing 3 with I throughout the function?

Describe alternatives you've considered

We could write our own functionality in TorchGeo, but figured this would be useful to the broader timm community.

Additional context

@isaaccorley @keves1 may also be interested in this.

Source: huggingface/pytorch-image-models