Purchased the Axera AX8850 and having problems
-
Hello,
I just purchased the Axera AX8850 and am having problems running the model Qwen3-1.7B in the directions on the M5 website. when run I receive the following errors:[json.exception.parse_error.101] parse error at line 1, column 1: attempting to parse an empty input; check that your input string or stream contains the expected JSON
[E][ Init][ 65]: get uid failed, try again 1/10
[json.exception.parse_error.101] parse error at line 1, column 1: attempting to parse an empty input; check that your input string or stream contains the expected JSON
[E][ Init][ 65]: get uid failed, try again 0/10
[json.exception.type_error.302] type must be number, but is null
[E][ Init][ 89]: get bos_id failed, try again 9/10
[json.exception.type_error.302] type must be number, but is null
[E][ Init][ 89]: get bos_id failed, try again 8/10
[json.exception.type_error.302] type must be number, but is null
[E][ Init][ 89]: get bos_id failed, try again 7/10At the end of the error I get the following:
bos_id: 0, eos_id: 0
terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_3::detail::type_error'
what(): [json.exception.type_error.302] type must be array, but is null
./run_qwen3_1.7b_int8_ctx_axcl_aarch64.sh: line 12: 1716 Aborted ./main_axcl_aarch64 --system_prompt "You are Qwen, created by Alibaba Cloud. You are a helpful assistant." --template_filename_axmodel "qwen3-1.7b-ax650/qwen3_p128_l%d_together.axmodel" --axmodel_num 28 --url_tokenizer_model "http://127.0.0.1:12345" --filename_post_axmodel qwen3-1.7b-ax650/qwen3_post.axmodel --filename_tokens_embed qwen3-1.7b-ax650/model.embed_tokens.weight.bfloat16.bin --tokens_embed_num 151936 --tokens_embed_size 2048 --use_mmap_load_embed 1 --live_print 1 --devices 0It looks like the tokenizer is running but I can't get the model running. I have tried several models from hugging's face and receive the same error. I am using a pi 5 with 16gb memory. Could you please let me know how to fix?
Thank you,
Mark -
Never mind, this is a model problem, I downloaded another model from hugging's face and it worked. Most of the models I tried from there do not work, I only got two of them to process. It was qwen2.5-7B and the Deepseek one. Both were a little slow, so I compared them using the CPU and the AX8850 and the CPU was slower, but not by much, This is on a pi 5 16GB board.
I also have a Kinara ara-2 to work on with 16GB memory and probably would be faster, but I am having problems converting models to its format. I got the pi 5 to recognize it via driver, I just need a converted model to try it. I also have a Hailo-10H coming, can't wait to test it for a project, its supposed to work better with Home Assistant.