Hi guys, I came across this thread while trying to fix some M5StickC issues and thought I could chime in with some useful info. You can use Arduino as an ESP-IDF component however, it currently only works up to v3.2.3 of ESP-IDF.
If you install v3.2.3 you can then use Arduino as a component from this repo here.
Hope this helps.