Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some changes for Docker running #974

Merged
merged 3 commits into from
Jun 12, 2023
Merged

Some changes for Docker running #974

merged 3 commits into from
Jun 12, 2023

Conversation

MrSedan
Copy link
Contributor

@MrSedan MrSedan commented Jun 12, 2023

It is causing errors when attempting to run Docker without headless due to a Tkinter display error. Additionally, if empty folders are not created locally, Docker will create them with root user owner, then the container will get "permission denied" error because of root user volumes.
Tkinter error without headless:
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/app/dreambooth_gui.py", line 366, in train_model output_message( File "/app/library/common_gui.py", line 83, in output_message msgbox(msg=msg, title=title) File "/usr/local/lib/python3.10/dist-packages/easygui/boxes/derived_boxes.py", line 230, in msgbox return buttonbox(msg=msg, File "/usr/local/lib/python3.10/dist-packages/easygui/boxes/button_box.py", line 95, in buttonbox bb = ButtonBox( File "/usr/local/lib/python3.10/dist-packages/easygui/boxes/button_box.py", line 147, in __init__ self.ui = GUItk(msg, title, choices, images, default_choice, cancel_choice, self.callback_ui) File "/usr/local/lib/python3.10/dist-packages/easygui/boxes/button_box.py", line 263, in __init__ self.boxRoot = tk.Tk() File "/usr/lib/python3.10/tkinter/__init__.py", line 2302, in __init__ self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use) _tkinter.TclError: no display name and no $DISPLAY environment variable

Permission denied error:

There was a problem when trying to write in your cache folder (/home/appuser/.cache/huggingface/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory.
prepare tokenizer
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /app/train_network.py:864 in <module>                                                            │
│                                                                                                  │
│   861 │   args = parser.parse_args()                                                             │
│   862 │   args = train_util.read_config_from_file(args, parser)                                  │
│   863 │                                                                                          │
│ ❱ 864 │   train(args)                                                                            │
│   865                                                                                            │
│                                                                                                  │
│ /app/train_network.py:92 in train                                                                │
│                                                                                                  │
│    89 │   │   args.seed = random.randint(0, 2**32)                                               │
│    90 │   set_seed(args.seed)                                                                    │
│    91 │                                                                                          │
│ ❱  92 │   tokenizer = train_util.load_tokenizer(args)                                            │
│    93 │                                                                                          │
│    94 │   # データセットを準備する                                                               │
│    95 │   blueprint_generator = BlueprintGenerator(ConfigSanitizer(True, True, True))            │
│                                                                                                  │
│ /app/library/train_util.py:2940 in load_tokenizer                                                │
│                                                                                                  │
│   2937 │   │   if args.v2:                                                                       │
│   2938 │   │   │   tokenizer = CLIPTokenizer.from_pretrained(original_path, subfolder="tokenize  │
│   2939 │   │   else:                                                                             │
│ ❱ 2940 │   │   │   tokenizer = CLIPTokenizer.from_pretrained(original_path)                      │
│   2941 │                                                                                         │
│   2942 │   if hasattr(args, "max_token_length") and args.max_token_length is not None:           │
│   2943 │   │   print(f"update token length: {args.max_token_length}")                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:1763 in          │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   1760 │   │   │   │   elif is_remote_url(file_path):                                            │
│   1761 │   │   │   │   │   resolved_vocab_files[file_id] = download_url(file_path, proxies=prox  │
│   1762 │   │   │   else:                                                                         │
│ ❱ 1763 │   │   │   │   resolved_vocab_files[file_id] = cached_file(                              │
│   1764 │   │   │   │   │   pretrained_model_name_or_path,                                        │
│   1765 │   │   │   │   │   file_path,                                                            │
│   1766 │   │   │   │   │   cache_dir=cache_dir,                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:409 in cached_file             │
│                                                                                                  │
│    406 │   user_agent = http_user_agent(user_agent)                                              │
│    407 │   try:                                                                                  │
│    408 │   │   # Load from URL or cache if already cached                                        │
│ ❱  409 │   │   resolved_file = hf_hub_download(                                                  │
│    410 │   │   │   path_or_repo_id,                                                              │
│    411 │   │   │   filename,                                                                     │
│    412 │   │   │   subfolder=None if len(subfolder) == 0 else subfolder,                         │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py:120 in _inner_fn    │
│                                                                                                  │
│   117 │   │   if check_use_auth_token:                                                           │
│   118 │   │   │   kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__name__, has_token=ha   │
│   119 │   │                                                                                      │
│ ❱ 120 │   │   return fn(*args, **kwargs)                                                         │
│   121 │                                                                                          │
│   122 │   return _inner_fn  # type: ignore                                                       │
│   123                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1131 in hf_hub_download │
│                                                                                                  │
│   1128 │   │   raise ValueError(f"Invalid repo type: {repo_type}. Accepted repo types are: {str  │
│   1129 │                                                                                         │
│   1130 │   storage_folder = os.path.join(cache_dir, repo_folder_name(repo_id=repo_id, repo_type  │
│ ❱ 1131 │   os.makedirs(storage_folder, exist_ok=True)                                            │
│   1132 │                                                                                         │
│   1133 │   # cross platform transcription of filename, to be used as a local file path.          │
│   1134 │   relative_filename = os.path.join(*filename.split("/"))                                │
│                                                                                                  │
│ /usr/lib/python3.10/os.py:215 in makedirs                                                        │
│                                                                                                  │
│    212 │   │   head, tail = path.split(head)                                                     │
│    213 │   if head and tail and not path.exists(head):                                           │
│    214 │   │   try:                                                                              │
│ ❱  215 │   │   │   makedirs(head, exist_ok=exist_ok)                                             │
│    216 │   │   except FileExistsError:                                                           │
│    217 │   │   │   # Defeats race condition when another thread created the path                 │
│    218 │   │   │   pass                                                                          │
│                                                                                                  │
│ /usr/lib/python3.10/os.py:215 in makedirs                                                        │
│                                                                                                  │
│    212 │   │   head, tail = path.split(head)                                                     │
│    213 │   if head and tail and not path.exists(head):                                           │
│    214 │   │   try:                                                                              │
│ ❱  215 │   │   │   makedirs(head, exist_ok=exist_ok)                                             │
│    216 │   │   except FileExistsError:                                                           │
│    217 │   │   │   # Defeats race condition when another thread created the path                 │
│    218 │   │   │   pass                                                                          │
│                                                                                                  │
│ /usr/lib/python3.10/os.py:225 in makedirs                                                        │
│                                                                                                  │
│    222 │   │   if tail == cdir:           # xxx/newdir/. exists if xxx/newdir exists             │
│    223 │   │   │   return                                                                        │
│    224 │   try:                                                                                  │
│ ❱  225 │   │   mkdir(name, mode)                                                                 │
│    226 │   except OSError:                                                                       │
│    227 │   │   # Cannot rely on checking for EEXIST, since the operating system                  │
│    228 │   │   # could give priority to other errors like EACCES or EROFS                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
PermissionError: [Errno 13] Permission denied: '/home/appuser/.cache/huggingface'
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/bin/accelerate:8 in <module>                                                          │
│                                                                                                  │
│   5 from accelerate.commands.accelerate_cli import main                                          │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(main())                                                                         │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/commands/accelerate_cli.py:45 in main         │
│                                                                                                  │
│   42 │   │   exit(1)                                                                             │
│   43 │                                                                                           │
│   44 │   # Run                                                                                   │
│ ❱ 45 │   args.func(args)                                                                         │
│   46                                                                                             │
│   47                                                                                             │
│   48 if __name__ == "__main__":                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/commands/launch.py:1104 in launch_command     │
│                                                                                                  │
│   1101 │   elif defaults is not None and defaults.compute_environment == ComputeEnvironment.AMA  │
│   1102 │   │   sagemaker_launcher(defaults, args)                                                │
│   1103 │   else:                                                                                 │
│ ❱ 1104 │   │   simple_launcher(args)                                                             │
│   1105                                                                                           │
│   1106                                                                                           │
│   1107 def main():                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/commands/launch.py:567 in simple_launcher     │
│                                                                                                  │
│    564 │   process = subprocess.Popen(cmd, env=current_env)                                      │
│    565 │   process.wait()                                                                        │
│    566 │   if process.returncode != 0:                                                           │
│ ❱  567 │   │   raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)       │
│    568                                                                                           │
│    569                                                                                           │
│    570 def multi_gpu_launcher(args):                                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

@bmaltais bmaltais changed the base branch from master to dev2 June 12, 2023 22:31
@bmaltais
Copy link
Owner

THank you for the code contribution.

@bmaltais bmaltais merged commit 5998ea2 into bmaltais:dev2 Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants