Liemph, thanks for sharing your observations.
I am totally new user of M5Stack Fire. My goal is to use it for teaching and for that reason, I would like to keep it simple and use only UIFlow and not to use Arduino or external Python tools.
In your case, why did you prefer Arduino over UIFlow?
My experience was that after many hours of changing the IDE versions and M5SF firmware, I was not able to download any code. After switching to flow.m5stack.com and selecting the latest M5SF firmware I was able to download and do it without USB cable.
PS. In your case study, I believe that the explanation is in the hardware reset when you are changing the bases.