I started from https://github.com/m5stack/esp-adf.git

It is probably better not to connect PIN 1 of the NS4148 but pin 3 the buck converter EA3036 to pin 5 since otherwise you have a bigger power consumption. You could also remove R23 in that case.