Suggestion: store decoded dataset images in RAM one time only instead of before every epoch

Search before asking

• I have searched the RF-DETR issues and found no similar feature requests.

Description

Hello all,

I checked and understood that the dataset images are read, decoded at every training epochs. This pipeline causes long training time and may deteriorate HDD as well (i guess my case).

Improvement: if we have sufficient RAM as my case, we should load the dataset one time only, decode and store them in RAM for the training process --> make the training faster

My suggestion:

1. Make a new training parameter called use_ram_cache:bool to let the user decide if they want to use RAM as a cache to load and store the dataset onetime only (set True) or keep reading dataset before every epoch as the current way (set False).
2. Live monitor free memory of RAM in rich progress bar

src/rfdetr/training/callbacks/gpu_memory_progress_bar.py:

import psutil                                #for RAM calculation

def get_metrics(self, trainer: Trainer, pl_module: LightningModule) -> _Metrics:
    ...
        free_ram= f"{(psutil.virtual_memory().available / 1024**3):.2f}/{(psutil.virtual_memory().total / 1024**3):.2f} GB_RAM"
    ...

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!