Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NotFoundError #17

Open
VirgilChen97 opened this issue Mar 18, 2018 · 3 comments
Open

NotFoundError #17

VirgilChen97 opened this issue Mar 18, 2018 · 3 comments

Comments

@VirgilChen97
Copy link

VirgilChen97 commented Mar 18, 2018

python reduce_model.py --model-input /home/virgil/Desktop/F2F/pix2pix-tensorflow/face2face-model --model-output face2face-reduced-model
2018-03-18 15:21:21.466217: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-18 15:21:21.466534: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: GeForce GTX 970 major: 5 minor: 2 memoryClockRate(GHz): 1.253
pciBusID: 0000:01:00.0
totalMemory: 3.95GiB freeMemory: 3.70GiB
2018-03-18 15:21:21.466552: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-03-18 15:21:21.652455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3436 MB memory) -> physical GPU (device: 0, name: GeForce GTX 970, pci bus id: 0000:01:00.0, compute capability: 5.2)
2018-03-18 15:21:21.761345: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key generator/decoder_1/deconv/filter not found in checkpoint
Traceback (most recent call last):
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1361, in _do_call
return fn(*args)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1340, in _run_fn
target_list, status, run_metadata)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 516, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_7 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_12_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "reduce_model.py", line 215, in
saver.restore(sess, checkpoint)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1755, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 905, in run
run_metadata_ptr)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1137, in _run
feed_dict_tensor, options, run_metadata)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1355, in _do_run
options, run_metadata)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1374, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_7 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_12_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Caused by op 'save/RestoreV2', defined at:
File "reduce_model.py", line 213, in
saver = tf.train.Saver()
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1293, in init
self.build()
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1302, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1339, in _build
build_save=build_save, build_restore=build_restore)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 796, in _build_internal
restore_sequentially, reshape)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 449, in _AddRestoreOps
restore_sequentially)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 847, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1030, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3271, in create_op
op_def=op_def)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1650, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

NotFoundError (see above for traceback): Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_7 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_12_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

As the same there are some problems when using reduce_model.py
How can I solve such problem?

@VirgilChen97
Copy link
Author

I've found the problem.
This version of face2face based on the April 2017 version of pix2pix which is based on tensorflow 1.0.0, but in December, pix2pix switched to tensorflow 1.4.1. clone the april version of pix2pix will solve the problem.

@AlecHaring
Copy link

So Tensorflow 1.0.0 should be installed when generating the model? I tried using Tensorflow 1.2.1 but it doesn't seem to want to use my GPU. Would I have to downgrade CUDA? If so, to what version? CUDA 8.0? I've been trying trial an error the past 2 days and can't seem to get over this hurdle. Having the same exact issue as OP

@noorkhokhar99
Copy link

So Tensorflow 1.0.0 should be installed when generating the model? I tried using Tensorflow 1.2.1 but it doesn't seem to want to use my GPU. Would I have to downgrade CUDA? If so, to what version? CUDA 8.0? I've been trying trial an error the past 2 days and can't seem to get over this hurdle. Having the same exact issue as OP

easly installation cuda and cnndn https://github.com/noorkhokhar99/Installation-window-10-tensorflow-Gpu/blob/master/cmd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants