[GraphX] initialmessage for pagerank should be 1.0

[luyi0619]

In the first round, pagerank value for each vertex should be 1.0, so the initialmessage should be set to 1.0, right?

[AmplabJenkins]

Can one of the admins verify this patch?

[luyi0619]

In my opinion, if the numIter is set to 0, then call Pregel, the pagerank value of each vertex should be 1, which is as the same as the traditional definition of pagerank.
Therefore, i think initialmessage should be set to 1, in order to let messages be correct which should be sent before running the pregel algorithm.

[ankurdave]

ok to test

cc @jegonzal

[ankurdave]

I don't think this will give the expected results. Consider the graph 1 --> 2. We expect PageRank to assign vertex 2 a higher rank than vertex 1, and currently GraphX does assign 1 the rank 0.15 and 2 the rank 0.2775. However, with this change GraphX would assign both 1 and 2 the same rank, 1.0.

[luyi0619]

Hi ankurdave,

In the graph you given(1-->2), I think your pagerank implementation will give the same results.

when the initalvalue is 0.0

a b
0 0
0.15 0.15
0.15 0.2775

when the initalvalue is 1.0

1 1
0.15 1
0.15 0.2775

Actually, there is another problem maybe we should handle if there is 0-outdegree vertices in the graph(e.g. 2). One solution is to distribute their pagerank values to all the vertices uniformly, but the initialvalue should be one.

For example,

a b
1 1
0.575 1.425
0.755625 1.24438
0.678859 1.32114
0.711485 1.28852

Omit some iterations.

0.701756 1.29824
0.701754 1.29825
0.701755 1.29825

In my understanding, if there is no 0-outdegree vertices in a graph, the sum of pagerank values from all vertices should remain the same. However, in the graphX implementation, this is not true.

In a word, I still think the initialvalue should be 1.0, you could also refer to graphlab's implementation.

52th line of https://github.com/graphlab-code/graphlab/blob/master/toolkits/graph_analytics/pagerank.cpp

[pwendell]

@luyi0619 can you add [GraphX] to the title of this pull request? It's not getting sorted correctly as it stands.

[luyi0619]

sure, thanks for your advice

[SparkQA]

Can one of the admins verify this patch?

[rxin]

cc @jegonzal can you take a look at this?

[srowen]

Is this a WontFix at this stage or needs another look from @ankurdave or @jegonzal ?

[ankurdave]

I talked to @jegonzal and it seems this is the correct thing to do. I'll merge it pending Jenkins.

[ankurdave]

Jenkins, test this please.

[SparkQA]

Test build #27840 has started for PR 1128 at commit 6f055f9.

• This patch does not merge cleanly.

[ankurdave]

Oh, this PR was against the old version of static PageRank that used Pregel, while we now use a standalone implementation. I think it still has the same problem and the fix seems to be to change the initially-assigned value from resetProb to 1. We can close this issue and I'll open a new one.

[jegonzal]

Great! I agree with this proposal as well. I apologize for letting it sit so long.

[SparkQA]

Test build #27840 has finished for PR 1128 at commit 6f055f9.

• This patch fails some tests.
• This patch does not merge cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27840/
Test FAILed.

[srowen]

@luyi0619 mind closing this PR? It sounds like this can be pursued in a new PR.