MPII3D performance reproduction

[markkim1115]

Hi, Thanks for the excellent work and code release.

I tried to reproduce MEVA's MPII3D PA-MPJPE performance, but i couldn't.

With my pre-processed MPII3D data file (made from VIBE official code, kocabas et al. 2020 CVPR), I ran your evaluation code.

Below are my settings and little modifications based on your evaluation code.

1. I didn't evaluated about the video segment of which number of frames is less than 90.
2. I modified your overlapped chunk generation function("get_chunk_with_overlap"). Because mpii3d have a video segment of which number of frames is 90, and the function generates indices set twice( [(0, 90), (0, 90)] ) for the video segment. Even though it needs frame indices only for the 0~89( (0,90) ). This bug made error for that video segment and crashes program while error calculating operation. Below is the actual values of variables in the function.

(Pdb) num_frames
90
(Pdb) chunk_idxes
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
        16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
        32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
        48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
        64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,
        80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
       [ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
        16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
        32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
        48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
        64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,
        80, 81, 82, 83, 84, 85, 86, 87, 88, 89]])
(Pdb) chunck_selects
[(0, 90), (0, 90)]

So i modified function like below.

def get_chunk_with_overlap(num_frames, window_size = 90, overlap = 0):
    assert overlap % 2 == 0
    step = window_size - overlap 
    chunk_idxes = view_as_windows(np.array(range(num_frames)), window_size, step= step)
    chunk_supp = np.linspace(num_frames - window_size, num_frames-1, num = window_size).astype(int)
    chunk_idxes = np.concatenate((chunk_idxes, chunk_supp[None, ]))
    last_chunk = chunk_idxes[-1][:step][-1] - chunk_idxes[-2][:step][-1] + int(overlap/2)
    chunck_selects = get_chunk_selects(chunk_idxes, last_chunk, window_size= window_size, overlap=overlap)
    
    return chunk_idxes, chunck_selects

def get_chunk_with_overlap(num_frames, window_size = 90, overlap = 0):
    assert overlap % 2 == 0
    step = window_size - overlap 
    chunk_idxes = view_as_windows(np.array(range(num_frames)), window_size, step= step)
    chunk_supp = np.linspace(num_frames - window_size, num_frames-1, num = window_size).astype(int)
    
    if num_frames != 90:
        chunk_idxes = np.concatenate((chunk_idxes, chunk_supp[None, ]))
        last_chunk = chunk_idxes[-1][:step][-1] - chunk_idxes[-2][:step][-1] + int(overlap/2)
        chunck_selects = get_chunk_selects(chunk_idxes, last_chunk, window_size= window_size, overlap=overlap)
    else:
        chunck_selects = [(0, 90)]
    return chunk_idxes, chunck_selects

After that, evaluation code worked, but result is pretty low than your result in this repo.

---

'mpjpe': 99.91301986073121,
'mpjpe_pa': 66.27225724514501,
'vel_err': 15.167348631137918,
'acc_err': 14.774026845370512,
'auc': 0.6680740727485766,
'pck': 94.77570523202951

---

Would you advice for the evaluation on mpii3d for the reproduction? Thanks!

[ZhengyiLuo]

Hi!

Thank you for the interest! Let me look into this a little bit more and get back to you asap.

[wang-zm18]

Same question!

[wang-zm18]

Hello, did yuo solve this problem? @ZhengyiLuo