Settings to reproduce result

[edwardjjj]

Dear Authors,

I tried to reproduce your results by running the code with the following command

python main_llm_vis.py --split val --eval 1 --auto_gpu_config 0 \
-n 8 --num_eval_episodes 250 --load pretrained_models/llm_model.pt \
--use_gtsem 0 --num_local_steps 10

The success rate was only 0.522. Could you please share your settings to produce the same result in the paper? Thank you very much!

[yusirhhh]

@edwardjjj @ybgdgh Have you resolved this issue or trained the model? The results I obtained using the released model are as follows:

succ/spl/dtg: 0.529/0.246/4.094(2000),

[edwardjjj]

@edwardjjj @ybgdgh Have you resolved this issue or trained the model? The results I obtained using the released model are as follows:

succ/spl/dtg: 0.529/0.246/4.094(2000),

Hi, I have ran tests many times and never get the claimed results. Best was 0.52x

[2003jiahang]

@yusirhhh @edwardjjj
I want to know how long have you run the whole dataset.
I saw author run the model on the HM3D val,the whole dataset seem more than 2000 tasks？

[edwardjjj]

We ran several eval runs, each with 2000 episodes. The best result we got was 52.4% with feed forward method.

[2003jiahang]

@edwardjjj As far as I konw,the HM3D-val seem have more than 2000 tasks. Do I have misunderstanding?
By the way ,I also can not reproduce the result.

[xuai05]

How do you run the program code correctly? I encountered the following problems during the operation.
However, my habitat-lab can run normally. Do you encounter the following problems?
@edwardjjj

[edwardjjj]

How do you run the program code correctly? I encountered the following problems during the operation.

However, my habitat-lab can run normally. Do you encounter the following problems?

@edwardjjj

Hi, I think that's a habitat-lab version issue. You need to install the correct version.

[xuai05]

I installed it from habitat-lab official website, url=https://github.com/facebookresearch/habitat-lab, can you tell me what version you are using?
@edwardjjj
My version is as follows:

[edwardjjj]

I installed it from habitat-lab official website, url=https://github.com/facebookresearch/habitat-lab, can you tell me what version you are using?

@edwardjjj

My version is as follows:

I think it was habitat-challenge-2022, check the tags in the habitat-lab repo.