live motion tracking with multiple m5stickc in unity [QUESTION]
-
Hey guys,
I want to use several m5stickc's to track the upper body movement.
The movement data should be streamed live into unity and applied to an armature.
The goal is to get somewhat accurate rotation and location data of the upper limb joints.Is this possible?
Kind regards,
Tobi -
It should be possible. You will need to connect the stickC’s to a server to receive the data and send it to unity and then something in unity to receive the data a deal with it. You can make a basic tracker in UIFlow which will record movement and transmit the values.
-
Yeah, with UIflow GetAccX blocks are not a problem, but sending data via socket connection? Could be solved easily with wifi <>uart esp32 mode but Im not sure its possible with uiflow
-
I started to use cloudmqtt and uiflow to send sensor data to my pc terminal via mqtt.
I read that there is a mqtt lib for unity. So will try to research that. :) -
@shintobi that is a good idea but for a more local version, you could used an M5Stack as a hub that connects to the pc over uart and have the sticks send the data to the stack.
UIFlow has peer to peer connections between units. -
@ajb2k3 my goal is to make the communication wireless.
do you mean i can get data from multiple devices in UIFlow at the same time? -
Have you considered using Bluetooth? I have connected a single Bluetooth tracking device (not from M5) to Unity without too much trouble. I believe Bluetooth supports up 7 clients connected at once.
-
@wikistik actually an interesting idea. do you also have experience connecting several m5s via bluetooth?
-
@shintobi no I don't
-
Okay, just got sent this link.
https://randomnerdtutorials.com/esp32-esp8266-raspberry-pi-lamp-server/ -
@ajb2k3 damn. i just randomly got the thought to check the forum and see you posted 2 mins ago. crazy...
Thanks i will check it out -
@shintobi only got the email an 10mins befor posting on the forum
-
I remember a Japanese user posting a video on twitter about using the M5Stack as a controller in unity. I found his slides its all in Japanese I'm afraid and not possible to copy past the text to translate but here's the link https://speakerdeck.com/naninunenoy/m5stack-with-unity in the slides he mentions a bluetooth library called grayblue, might be worth a shot checking that out. I also found these other two blog posts covering the subject again in Japanese but this time at least you should be able to copy and paste the text to translate https://qiita.com/Zeni-Y/items/1d0e545259fbcd9381c8 and http://prince9.hatenablog.com/entry/2018/09/26/055809
-
@shintobi , just sent you a voucher to our Unity MQTT asset.
-
@ajb2k3 for this use case I would recommend streaming IMU data to InfluxDB. Seems more appropriate for limb tracking.
-
@lukasmaximus , below is some code I used on PYCOM hardware and WebBluetooth. I ran into issues when getting characteristic notifications faster than 1/2 sec. Not sure if it was an IDF GATT issue or WebBluetooth or PYCOM implementation. But it did kinda work.
-
@cmisztur thanks for sharing
-
@cmisztur the unity mqtt asset looks nice. but where can i put port / user / pwd? :D
i am using cloudmqtt as my broker. -
@lukasmaximus will check it out. thanks!
-
@shintobi check out some of the scenes like ping-pong. It’s in a script on a GO.
