【C++】throw exception when use MXDataIter("CSVIter") to train

[zxy110]

Hello, I want to user CSVIter in my cpp program, but I got an exception at
MXDataIterBeforeFirst(blob_ptr_->handle_);
in io.hpp
I don't know whether it's caused by the shapes I set.
I've use the mnist datasets, Here is my code.

#include <chrono>
#include "utils.h"
#include "mxnet-cpp/MxNetCpp.h"
using namespace mxnet::cpp;

Symbol mlp(const std::vector<int> &layers) {
	auto x = Symbol::Variable("X");
	auto label = Symbol::Variable("label");

	std::vector<Symbol> weights(layers.size());
	std::vector<Symbol> biases(layers.size());
	std::vector<Symbol> outputs(layers.size());
	for (size_t i = 0; i < layers.size(); ++i) {
		weights[i] = Symbol::Variable("w" + std::to_string(i));
		biases[i] = Symbol::Variable("b" + std::to_string(i));
		Symbol fc = FullyConnected(
			i == 0 ? x : outputs[i - 1],  // data
			weights[i],
			biases[i],
			layers[i]);
		outputs[i] = i == layers.size() - 1 ? fc : Activation(fc, ActivationActType::kRelu);
	}
return SoftmaxOutput(outputs.back(), label);
}

int main(int argc, char** argv) {
	const int image_size = 28;
	const std::vector<int> layers{ 128, 64, 10 };
	const int batch_size = 100;
	const int max_epoch = 10;
	const float learning_rate = 0.1;
	const float weight_decay = 1e-2;

	std::vector<std::string> data_files = { "D:/mxnet/test3/some_test/mxnet_test/mxnet-	cvsiter/data/mnist/mnist_iter_train_data.csv",
	"D:/mxnet/test3/some_test/mxnet_test/mxnet-cvsiter/data/mnist/mnist_iter_train_label.csv",
	"D:/mxnet/test3/some_test/mxnet_test/mxnet-cvsiter/data/mnist/mnist_iter_test_data.csv",
	"D:/mxnet/test3/some_test/mxnet_test/mxnet-cvsiter/data/mnist/mnist_iter_test_label.csv"
};

auto train_iter = MXDataIter("CSVIter");
setDataIter(&train_iter, "Train", data_files, batch_size, 0);			//shuffle = 1

auto val_iter = MXDataIter("CSVIter");
setDataIter(&val_iter, "Label", data_files, batch_size, 0);				//shuffle = 0
auto net = mlp(layers);

Context ctx = Context::cpu();  // Use CPU for training


std::map<std::string, NDArray> args;
args["X"] = NDArray(Shape(batch_size, image_size*image_size), ctx);
args["label"] = NDArray(Shape(batch_size), ctx);
// Let MXNet infer shapes other parameters such as weights
net.InferArgsMap(ctx, &args, args);

// Initialize all parameters with uniform distribution U(-0.01, 0.01)
auto initializer = Uniform(0.01);
for (auto& arg : args) {
	// arg.first is parameter name, and arg.second is the value
	initializer(arg.first, &arg.second);
}

// Create sgd optimizer
Optimizer* opt = OptimizerRegistry::Find("sgd");
opt->SetParam("rescale_grad", 1.0 / batch_size)
	->SetParam("lr", learning_rate)
	->SetParam("wd", weight_decay);

// Create executor by binding parameters to the model
auto *exec = net.SimpleBind(ctx, args);
auto arg_names = net.ListArguments();

// Start training
for (int iter = 0; iter < max_epoch; ++iter) {
	int samples = 0;
	train_iter.Reset();

	auto tic = std::chrono::system_clock::now();
	while (train_iter.Next()) {
		samples += batch_size;
		auto data_batch = train_iter.GetDataBatch();
		// Set data and label
		data_batch.data.CopyTo(&args["X"]);
		data_batch.label.CopyTo(&args["label"]);

		// Compute gradients
		exec->Forward(true);
		exec->Backward();
		// Update parameters
		for (size_t i = 0; i < arg_names.size(); ++i) {
			if (arg_names[i] == "X" || arg_names[i] == "label") continue;
			opt->Update(i, exec->arg_arrays[i], exec->grad_arrays[i]);
		}
	}
	auto toc = std::chrono::system_clock::now();

	Accuracy acc;
	val_iter.Reset();
	while (val_iter.Next()) {
		auto data_batch = val_iter.GetDataBatch();
		data_batch.data.CopyTo(&args["X"]);
		data_batch.label.CopyTo(&args["label"]);
		// Forward pass is enough as no gradient is needed when evaluating
		exec->Forward(false);
		acc.Update(data_batch.label, exec->outputs[0]);
	}
	float duration = std::chrono::duration_cast<std::chrono::milliseconds>
		(toc - tic).count() / 1000.0;
LG << "Epoch: " << iter << " " << samples / duration << " samples/sec Accuracy: " << acc.Get();
}

delete exec;
MXNotifyShutdown();
return 0;
}

[vdantu]

@mxnet-label-bot : [C++, Question]

[leleamol]

Can you please share how you are setting the parameters for the CSVIter iterator? To be precise, the implementation of "setDataIter()" used in the above code.

According to https://github.com/apache/incubator-mxnet/blob/master/src/io/iter_csv.cc, for CSVIter to work correctly, the parameters such as "data_csv", "data_shape",etc are required to be set.

The implementation of "setDataIter()" in the "mxnet-cpp/examples/utils.h" is specific to "MNISTIter". The function is setting the parameters (such as "image", "label", etc) that are specific to MNISTIter.

[leleamol]

You can try using the "CSVIter" as follows.

auto train_iter = MXDataIter("CSVIter")
.SetParam("data_csv", "./data/mnist_iter_train_data.csv")
.SetParam("data_shape", Shape(784,1))
.SetParam("label_csv", "./data/mnist_iter_train_label.csv")
.SetParam("label_shape", Shape(1,1))
.SetParam("batch_size", batch_size)
.SetParam("flat", 1)
.SetParam("shuffle",1)
.CreateDataIter();

auto val_iter = MXDataIter("CSVIter")
.SetParam("data_csv", "./data/mnist_iter_test_data.csv")
.SetParam("data_shape", Shape(784, 1))
.SetParam("label_csv", "./data/mnist_iter_test_label.csv")
.SetParam("label_shape", Shape(1,1))
.SetParam("batch_size", batch_size)
.SetParam("flat", 1)
.SetParam("shuffle",1)
.CreateDataIter();

In the above example "mnist_iter_train_data.csv" and "mnist_iter_test_data.csv" files contain the input features as 784 columns and "mnist_iter_train_label.csv", "mnist_iter_test_label.csv" as single column of labels.

[lupesko]

@leleamol

Can you please share how you are setting the parameters for the CSVIter iterator? To be precise, the implementation of "setDataIter()" used in the above code.

I think this function is part of the example, in example/utils.h

Would be great is if you can try to reproduce the issue @zxy110 is facing, and either fix the example, or show how the usage is incorrect.

[leleamol]

@lupesko the 'setDataIter' function in example/utils.h is hard coded for MNISTIter.
I have written an example to demonstrate CSVIter. The pr is out for review.

PR reference:
#12636

[lupesko]

@leleamol does #12636 solve the issue here?

[leleamol]

@lupesko yes it does solve the issue. The example demonstrates how to use CSVIter.

[leleamol]

@zxy110
I have added an example to demonstrate the usage of CSVIter with the PR #12636
Please let me know if this example addresses your concern and helps you in implementing your use case, we can then mark this issue as 'closed'.

[leleamol]

@mxnet-label-bot : [pending-requester-info]

[zxy110]

@leleamol Yes I've solved the problem, thank you very much!