1. Have you tried profiling your code with pytorch to see what the bottleneck is?
2. If your goal is to have more control and knowledge over what's happening during training, you can modify a cust...
@pcarre-lab here the code is: https://github.com/25benjaminli/sam2lora
To be completely honest, I didn't find much difference fine tuning this version. The training speed seemed just about the s...