After a lot of testing I came to the following conclusion:

The problem was that I waited too long before enabling GPIO pin 12 in my code. This pin is used to keep the device powered by the battery.

I'm just speculating here, but it seems like there is a time frame in which you need to enable this pin, otherwise the device will not successfully boot.