I was experiencing the same issue and through trial and error I resolved it for me by removing the version 3.20 M5Stack board, quitting the Arduino IDE and deleting the username/Library/Arduino15 folder. I then restarted the IDE and installed the version 2.1.4 M5Stack board.
I then uninstalled the M5Stack library version 0.4.6 and reinstalled it. Has worked successfully on two silicon based Macs. (MacBook Pro M1 and MacStudio)