Yes, it’s very annoying. And really you end up using about ~80kb for firmware, leaving very little space for your own patch code.
You could also try to use the new bootloader and store code on QSPI - Creating a Bootloader via the Uart - #18 by shensley