Paper Result

[scy04]

Can you provide a full instruction on ShinyBlender dataset? I have no idea how to reproduce the paper result. And I am wondering why the data in your paper are inconsistent.

[scy04]

@Asparagus15 Your reply will be highly appreciated

[TangZJ]

I'm also having issues replicating the results. But i think that figure 1(a) result is based on per frame.

[ingra14m]

Here are my results using the latest commit. I used the command python train.py -s path/to/refnerf --eval -m output/refnerf/name -w --brdf_dim 0 --sh_degree -1 --lambda_predicted_normal 2e-1 --brdf_env 512 under 800x800 resolution for all the experiments.

Ref-NeRF

Scene	PSNR	SSIM	LPIPS(VGG)
ball	29.27	0.9563	0.1428
car	28.44	0.939	0.0475
coffee	31.02	0.9689	0.0855
helmet	28.13	0.9516	0.0902
teapot	43.58	0.9957	0.0107
toaster	23.93	0.9107	0.1024
Mean	30.73	0.954	0.0798

NSVF

Scene	PSNR	SSIM	LPIPS (VGG)
Bike	37.38	0.9917	0.0066
Lifestyle	27.36	0.9636	0.0509
Palace	36.55	0.9791	0.0198
Robot	37.00	0.9938	0.0082
Spaceship	32.61	0.9847	0.0158
Steamtrain	35.27	0.9903	0.0103
Toad	34.50	0.9795	0.0227
Wineholder	30.16	0.9657	0.0291
Average	33.85	0.981	0.0204

tbh, I don't think the quality of the shortest axis normal is sufficient for good reflection. The absence of GS's geometry makes it difficult to distinguish between environment and material through geometry (normal). I've even used the gt normal of refnerf to train models, but the improvement in geometry hardly helps the 3D-GS rendering, and the reflective part is still blurred.

Under the premise that differentiable surface rendering currently cannot surpass volumetric rendering, I truly believe that able-nerf is an astonishing method, except for being too slow (4 V100s for 3 days) and the interpretability of color tokens (maybe the backpack language model could help?) @TangZJ .