Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

train.rec test.rec for cifar100 #235

Closed
ranti-iitg opened this issue Oct 8, 2015 · 3 comments · Fixed by sociomantic-tsunami/mxnet#27
Closed

train.rec test.rec for cifar100 #235

ranti-iitg opened this issue Oct 8, 2015 · 3 comments · Fixed by sociomantic-tsunami/mxnet#27

Comments

@ranti-iitg
Copy link

where can i find train.rec and test.rec for cifar100

@antinucleon
Copy link
Contributor

The train.rec and test.rec is generated by original Alex's python format. We first dump images from the binary file, then pack them into rec file. Note you don't need to set resize parameter for im2rec in this case

A sample code I used before to dump images:

import os
import sys
import pickle
import csv
import numpy as np
from PIL import Image

if len(sys.argv) < 3:
    print "usage: convert cifar.pkl output_root_folder_path output_list_path"
    exit(-1)

cifar = pickle.load(file(sys.argv[1]))
root = sys.argv[2]
try:
    os.mkdir(root)
except:
    print "%s exists, ignore"

fo = csv.writer(open(sys.argv[3], "w"), lineterminator='\n', delimiter='\t')

data = cifar['data']
label = cifar['fine_labels']
path = cifar['filenames']

sz = data.shape[0]

for i in xrange(sz):
    if i % 1000 == 0:
        print i
    img = data[i]
    p = path[i]
    img = img.reshape((3, 32, 32))
    img = np.swapaxes(img, 0, 2)
    img = np.swapaxes(img, 0, 1)
    im = Image.fromarray(img)
    im.save(root + p)
    row = [i, label[i], p]
    fo.writerow(row)

@ranti-iitg
Copy link
Author

link to binary version of data http://www.cs.toronto.edu/~kriz/cifar-100-binary.tar.gz binary version of data, also can you share your trained model for cifar100 like cafe.

@wanderingpj
Copy link

Do you know what networks are suitable for cifar100 to get to a high testing accuracy?@ranti-iitg @antinucleon

eric-haibin-lin pushed a commit to eric-haibin-lin/mxnet that referenced this issue Dec 2, 2017
iblislin pushed a commit to iblislin/incubator-mxnet that referenced this issue Mar 18, 2018
fixed typo in comment
joseph-wakeling-sociomantic added a commit to joseph-wakeling-sociomantic/mxnet that referenced this issue Mar 23, 2018
https://github.com/apache/incubator-mxnet/releases/tag/1.1.0

A quite extended amount of manual work was needed here to resolve all
the conflicts, almost none due to the tsunami fork!

Some diffs seemed to survive the merge, and were fixed via manually
checking out the files in question from the upstream 1.1.0 tag.

* 3rdparty/cub ()...05eb57fa(05eb57fa) (1 commits)
  > Merge pull request sociomantic-tsunami#1 from ptrendx/update

* 3rdparty/googletest ()...release-1.8.0(ec44c6c) (1 commits)
  > Merge pull request apache#821 from mazong1123/master

* 3rdparty/openmp ()...37c7212(37c7212) (1 commits)
  > Merging r317115:

* 3rdparty/cub ()...05eb57fa(05eb57fa) (1 commits)
  > Merge pull request sociomantic-tsunami#1 from ptrendx/update

* 3rdparty/googletest ()...release-1.8.0(ec44c6c) (1 commits)
  > Merge pull request apache#821 from mazong1123/master

* 3rdparty/openmp ()...37c7212(37c7212) (1 commits)
  > Merging r317115:

* dmlc-core 87b7ffa(87b7ffa)...6389c10(6389c10) (20 commits)
  > Fix symbol demangling on stacktraces under Linux. (apache#356)
  > Add more environment variables to support 3rd-party S3 implementations (apache#354)
  > add qid as ranklib format (apache#317)
  > Make choice of archiver configurable (apache#352)
  > Fix warnings in unittest_lockfree.cc, fix travis CI (apache#347)
  (...)

* mshadow 2d7780c(2d7780c)...16ac8cd(16ac8cd) (5 commits)
  > Add fully qualified type on define default_real_t (apache#317)
  > fix AddTakeGradLargeBatch for CPU. close apache#235 (apache#316)
  > fix float16 min and max values (apache#315)
  > [Windows] make Packet::size constexpr (apache#313)
  > add GetRndEngine (apache#308)

* nnvm e4a138a(e4a138a)...7a052d6(7a052d6) (3 commits)
  > [SYMBOL] Add __iter__ and GetChildren for symbol (apache#268)
  > API call to get symbol output count (apache#270)
  > [FRONTEND] Fix mxnet multi outputs (apache#271)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants