Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support hdfs://namenode:port for hdfs object storage #3713

Merged
merged 7 commits into from
May 30, 2023
Merged

Support hdfs://namenode:port for hdfs object storage #3713

merged 7 commits into from
May 30, 2023

Conversation

tangyoupeng
Copy link
Contributor

@tangyoupeng tangyoupeng commented May 30, 2023 •
edited
Loading 

    <name>dfs.namenode.rpc-address.ns.namenode157</name>
    <value>hadoop02:8020</value>
  </property>

  <property>
    <name>dfs.ha.namenodes.ns</name>
    <value>namenode108,namenode157</value>
  </property>

@tangyoupeng tangyoupeng requested a review from timfeirg May 30, 2023 04:17
@tangyoupeng tangyoupeng marked this pull request as draft May 30, 2023 04:37
@tangyoupeng tangyoupeng marked this pull request as ready for review May 30, 2023 05:13
@tangyoupeng tangyoupeng requested a review from davies May 30, 2023 07:59
@davies davies merged commit d99abda into main May 30, 2023
@davies davies deleted the hdfs_schema branch May 30, 2023 09:23
@github-actions
Copy link

Mutate Test Report
pkg/meta/tkv_fdb_test.go-1-1: score:0.00 failed:0, passed:0, compile error:0, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709541

pkg/meta/benchmarks_test.go-1-1: score:0.00 failed:0, passed:0, compile error:0, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709314

pkg/usage/usage_test.go-1-1: score:0.23 failed:10, passed:3, compile error:1, out of coverage:3, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709611

pkg/object/object_storage_test.go-1-1: score:0.17 failed:10, passed:2, compile error:1, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709581

pkg/fs/fs_test.go-1-4: score:0.50 failed:17, passed:17, compile error:21, out of coverage:45, skip by comment:0, others:1
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709072

pkg/fs/fs_test.go-2-4: score:0.18 failed:55, passed:12, compile error:10, out of coverage:24, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709130

pkg/fuse/fuse_test.go-1-1: score:0.21 failed:49, passed:13, compile error:11, out of coverage:37, skip by comment:0, others:1
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709273

pkg/fs/fs_test.go-4-4: score:0.49 failed:43, passed:42, compile error:4, out of coverage:9, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709216

cmd/mount_test.go-2-4: score:1.00 failed:0, passed:51, compile error:2, out of coverage:8, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200708898

cmd/mount_test.go-1-4: score:1.00 failed:0, passed:52, compile error:9, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200708834

cmd/object_test.go-1-1: score:0.25 failed:71, passed:24, compile error:11, out of coverage:4, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709043

pkg/vfs/vfs_test.go-1-4: score:0.48 failed:50, passed:46, compile error:31, out of coverage:7, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709641

pkg/vfs/vfs_test.go-3-4: score:0.38 failed:73, passed:44, compile error:14, out of coverage:3, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709712

pkg/fs/fs_test.go-3-4: score:0.42 failed:46, passed:34, compile error:4, out of coverage:17, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709172

pkg/vfs/vfs_test.go-2-4: score:0.25 failed:96, passed:32, compile error:0, out of coverage:6, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709673

pkg/vfs/vfs_test.go-4-4: score:0.52 failed:50, passed:55, compile error:10, out of coverage:17, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709753

pkg/meta/sql_test.go-1-4: score:0.36 failed:148, passed:82, compile error:92, out of coverage:95, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709358

pkg/meta/sql_test.go-2-4: score:0.24 failed:133, passed:42, compile error:26, out of coverage:216, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709401

pkg/meta/sql_test.go-4-4: score:0.38 failed:166, passed:102, compile error:42, out of coverage:106, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709501

pkg/meta/sql_test.go-3-4: score:0.34 failed:217, passed:114, compile error:42, out of coverage:44, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5117504703/jobs/9200709446

Usage: https://github.com/juicedata/juicefs/blob/main/.github/scripts/mutate/how_to_use_mutate_test.md

@github-actions
Copy link

Mutate Test Report
pkg/meta/tkv_fdb_test.go-1-1: score:0.00 failed:0, passed:0, compile error:0, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603832

pkg/meta/benchmarks_test.go-1-1: score:0.00 failed:0, passed:0, compile error:0, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603107

pkg/usage/usage_test.go-1-1: score:0.23 failed:10, passed:3, compile error:1, out of coverage:3, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204604152

pkg/object/object_storage_test.go-1-1: score:0.17 failed:10, passed:2, compile error:1, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603963

pkg/fs/fs_test.go-1-4: score:0.50 failed:17, passed:17, compile error:21, out of coverage:45, skip by comment:0, others:1
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602495

pkg/fs/fs_test.go-2-4: score:0.18 failed:55, passed:12, compile error:10, out of coverage:24, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602613

pkg/fuse/fuse_test.go-1-1: score:0.21 failed:49, passed:13, compile error:11, out of coverage:37, skip by comment:0, others:1
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602989

cmd/mount_test.go-4-4: score:1.00 failed:0, passed:16, compile error:4, out of coverage:0, skip by comment:0, others:38
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602238

pkg/fs/fs_test.go-4-4: score:0.49 failed:43, passed:42, compile error:4, out of coverage:9, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602875

cmd/mount_test.go-2-4: score:1.00 failed:0, passed:51, compile error:2, out of coverage:8, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204601966

cmd/mount_test.go-1-4: score:1.00 failed:0, passed:52, compile error:9, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204601824

cmd/mount_test.go-3-4: score:1.00 failed:0, passed:59, compile error:2, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602100

pkg/vfs/vfs_test.go-1-4: score:0.48 failed:50, passed:46, compile error:31, out of coverage:7, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204604282

cmd/object_test.go-1-1: score:0.25 failed:71, passed:24, compile error:11, out of coverage:4, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602381

pkg/vfs/vfs_test.go-3-4: score:0.38 failed:73, passed:44, compile error:14, out of coverage:3, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204604573

pkg/vfs/vfs_test.go-2-4: score:0.25 failed:96, passed:32, compile error:0, out of coverage:6, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204604437

pkg/fs/fs_test.go-3-4: score:0.44 failed:45, passed:35, compile error:4, out of coverage:17, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204602746

pkg/vfs/vfs_test.go-4-4: score:0.52 failed:50, passed:55, compile error:10, out of coverage:17, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204604709

pkg/meta/sql_test.go-1-4: score:0.35 failed:150, passed:81, compile error:92, out of coverage:94, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603249

pkg/meta/sql_test.go-2-4: score:0.24 failed:133, passed:43, compile error:26, out of coverage:215, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603401

pkg/meta/sql_test.go-4-4: score:0.39 failed:164, passed:104, compile error:42, out of coverage:106, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603686

pkg/meta/sql_test.go-3-4: score:0.34 failed:219, passed:112, compile error:42, out of coverage:44, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119360085/jobs/9204603535

Usage: https://github.com/juicedata/juicefs/blob/main/.github/scripts/mutate/how_to_use_mutate_test.md

@github-actions
Copy link

Mutate Test Report
pkg/meta/benchmarks_test.go-1-1: score:0.00 failed:0, passed:0, compile error:0, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723530

pkg/usage/usage_test.go-1-1: score:0.23 failed:10, passed:3, compile error:1, out of coverage:3, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204724588

pkg/object/object_storage_test.go-1-1: score:0.17 failed:10, passed:2, compile error:1, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204724432

pkg/meta/tkv_fdb_test.go-1-1: score:0.00 failed:0, passed:0, compile error:0, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204724274

pkg/fs/fs_test.go-1-4: score:0.50 failed:17, passed:17, compile error:21, out of coverage:45, skip by comment:0, others:1
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204722811

pkg/fs/fs_test.go-2-4: score:0.17 failed:58, passed:12, compile error:10, out of coverage:21, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204722950

cmd/mount_test.go-4-4: score:1.00 failed:0, passed:16, compile error:4, out of coverage:0, skip by comment:0, others:38
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204722496

pkg/fs/fs_test.go-4-4: score:0.49 failed:43, passed:42, compile error:4, out of coverage:9, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723236

pkg/fuse/fuse_test.go-1-1: score:0.21 failed:49, passed:13, compile error:11, out of coverage:37, skip by comment:0, others:1
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723377

cmd/mount_test.go-1-4: score:1.00 failed:0, passed:52, compile error:9, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204721942

cmd/mount_test.go-2-4: score:1.00 failed:0, passed:51, compile error:2, out of coverage:8, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204722135

cmd/mount_test.go-3-4: score:1.00 failed:0, passed:59, compile error:2, out of coverage:0, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204722333

cmd/object_test.go-1-1: score:0.25 failed:71, passed:24, compile error:11, out of coverage:4, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204722676

pkg/vfs/vfs_test.go-1-4: score:0.48 failed:50, passed:46, compile error:31, out of coverage:7, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204724766

pkg/vfs/vfs_test.go-3-4: score:0.38 failed:73, passed:44, compile error:14, out of coverage:3, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204725065

pkg/fs/fs_test.go-3-4: score:0.44 failed:45, passed:35, compile error:4, out of coverage:17, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723110

pkg/vfs/vfs_test.go-2-4: score:0.25 failed:96, passed:32, compile error:0, out of coverage:6, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204724925

pkg/vfs/vfs_test.go-4-4: score:0.52 failed:50, passed:55, compile error:10, out of coverage:17, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204725228

pkg/meta/sql_test.go-1-4: score:0.35 failed:149, passed:81, compile error:92, out of coverage:95, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723685

pkg/meta/sql_test.go-2-4: score:0.25 failed:132, passed:43, compile error:26, out of coverage:216, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723824

pkg/meta/sql_test.go-4-4: score:0.39 failed:164, passed:104, compile error:42, out of coverage:106, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204724120

pkg/meta/sql_test.go-3-4: score:0.34 failed:218, passed:113, compile error:42, out of coverage:44, skip by comment:0, others:0
Job detail: https://github.com/juicedata/juicefs/actions/runs/5119407835/jobs/9204723979

Usage: https://github.com/juicedata/juicefs/blob/main/.github/scripts/mutate/how_to_use_mutate_test.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davies davies davies approved these changes

@timfeirg timfeirg timfeirg approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tangyoupeng @davies @timfeirg