mxnet.model.server custom Service model.get_outputs() returns only one result

[crazyleg]

Description

When using mxnet.model.server custom Service model.get_outputs() returns only one result, while data supplied to self.mx_model.forward(DataBatch(data), is_train=False) has a list of many.

Environment info (Required)

I assume not required here
I'am using Python.

Minimum reproducible example

def _inference(self, data):
   
    data = [item.as_in_context(self.ctx) for item in data]
    #print(len(data)) gives 100
    #print(len(DataBatch(data).data)) gives 100
  
    self.mx_model.forward(DataBatch(data), is_train=False)
    q = self.mx_model.get_outputs()

    #print(len(q)) gives 1

    return self.mx_model.get_outputs()

What have you tried to solve it?

Currently I am iterating over data and wrap each separate sample into DataBatch and run separate inference. While this works, I assume that running forward with full DataBatch will give better performance.

How should I do model.forward(DataBatch(data)) to get a list of outputs that is equal to len(data) ? Looks like a problem.

[rajanksin]

@crazyleg Please post your question on https://github.com/awslabs/mxnet-model-server/issues
to get better response for this issue.

@reminisce Please tag : Question