Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Offset chunk of spark writer got wrong value and output location #62

Closed
1 task done
acezen opened this issue Jan 5, 2023 · 1 comment
Closed
1 task done

[Bug]: Offset chunk of spark writer got wrong value and output location #62

acezen opened this issue Jan 5, 2023 · 1 comment
Assignees
@acezen
Labels
bug Something isn't working
Milestone
Release v0.1.0

Comments

@acezen
Copy link
Contributor

acezen commented Jan 5, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

  • Offset chunk file's path output by spark writer is not compatible with the path get from edge info:

run mvn test -Dsuites='com.alibaba.graphar.WriterSuite test edge writer with vertex table and edge table

the offset0 output path is /tmp/edge/person_knows_person/ordered_by_source/offset/part0
but [getAdjListOffsetFilePath] method of edge info return /tmp/edge/person_knows_person/ordered_by_source/offset0
https://github.com/alibaba/GraphAr/blob/0991064e3f5a5844d453d2743bc2b03dc65fdf14/spark/src/main/scala/com/alibaba/graphar/EdgeInfo.scala#L291

Expected Behavior

  • path: offset output path should compatible with edge info
  • offset value should be the real offset value, not edge count.

Minimal Reproducible Example

cd spark
mvn test -Dsuites='com.alibaba.graphar.WriterSuite test edge writer with vertex table and edge table

Environment

  • Operating system: MacOS
  • GraphAr version:
    v0.1.0

Link to GraphAr Logs

No response

Further Information

No response
@acezen acezen self-assigned this Jan 5, 2023
@acezen acezen added the bug Something isn't working label Jan 5, 2023
@acezen acezen moved this to In Progress in alibaba/GraphAr Jan 5, 2023
@acezen acezen added this to the Release v0.1.0 milestone Jan 5, 2023
@acezen acezen mentioned this issue Jan 5, 2023
Merged
9 tasks
@acezen acezen moved this from In Progress to In Review in alibaba/GraphAr Jan 5, 2023
@acezen
Copy link
Contributor Author

acezen commented Jan 5, 2023

solved by pull request #63

@acezen acezen closed this as completed Jan 5, 2023
@github-project-automation github-project-automation bot moved this from In Review to Done in alibaba/GraphAr Jan 5, 2023
@acezen acezen mentioned this issue Jan 10, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@acezen acezen

Labels
bug Something isn't working
Projects
None yet
Milestone
Release v0.1.0
Development

No branches or pull requests
1 participant
@acezen