Skip to content

Commit b97070e

Browse files
committed
[Doc][GraphX] Remove Motivation section and did some minor update.
1 parent 90a6a46 commit b97070e

File tree

1 file changed

+7
-70
lines changed

1 file changed

+7
-70
lines changed

docs/graphx-programming-guide.md

Lines changed: 7 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -57,77 +57,15 @@ title: GraphX Programming Guide
5757

5858
# Overview
5959

60-
GraphX is the new (alpha) Spark API for graphs and graph-parallel computation. At a high level,
61-
GraphX extends the Spark [RDD](api/scala/index.html#org.apache.spark.rdd.RDD) by introducing the
62-
[Resilient Distributed Property Graph](#property_graph): a directed multigraph with properties
60+
GraphX is a new component in Spark for graphs and graph-parallel computation. At a high level,
61+
GraphX extends the Spark [RDD](api/scala/index.html#org.apache.spark.rdd.RDD) by introducing a
62+
new [Graph](#property_graph) abstraction: a directed multigraph with properties
6363
attached to each vertex and edge. To support graph computation, GraphX exposes a set of fundamental
6464
operators (e.g., [subgraph](#structural_operators), [joinVertices](#join_operators), and
65-
[aggregateMessages](#aggregateMessages)) as well as an optimized variant of the [Pregel](#pregel) API. In
66-
addition, GraphX includes a growing collection of graph [algorithms](#graph_algorithms) and
65+
[aggregateMessages](#aggregateMessages)) as well as an optimized variant of the [Pregel](#pregel) API. In addition, GraphX includes a growing collection of graph [algorithms](#graph_algorithms) and
6766
[builders](#graph_builders) to simplify graph analytics tasks.
6867

6968

70-
## Motivation
71-
72-
From social networks to language modeling, the growing scale and importance of
73-
graph data has driven the development of numerous new *graph-parallel* systems
74-
(e.g., [Giraph](http://giraph.apache.org) and
75-
[GraphLab](http://graphlab.org)). By restricting the types of computation that can be
76-
expressed and introducing new techniques to partition and distribute graphs,
77-
these systems can efficiently execute sophisticated graph algorithms orders of
78-
magnitude faster than more general *data-parallel* systems.
79-
80-
<p style="text-align: center;">
81-
<img src="img/data_parallel_vs_graph_parallel.png"
82-
title="Data-Parallel vs. Graph-Parallel"
83-
alt="Data-Parallel vs. Graph-Parallel"
84-
width="50%" />
85-
<!-- Images are downsized intentionally to improve quality on retina displays -->
86-
</p>
87-
88-
However, the same restrictions that enable these substantial performance gains also make it
89-
difficult to express many of the important stages in a typical graph-analytics pipeline:
90-
constructing the graph, modifying its structure, or expressing computation that spans multiple
91-
graphs. Furthermore, how we look at data depends on our objectives and the same raw data may have
92-
many different table and graph views.
93-
94-
<p style="text-align: center;">
95-
<img src="img/tables_and_graphs.png"
96-
title="Tables and Graphs"
97-
alt="Tables and Graphs"
98-
width="50%" />
99-
<!-- Images are downsized intentionally to improve quality on retina displays -->
100-
</p>
101-
102-
As a consequence, it is often necessary to be able to move between table and graph views.
103-
However, existing graph analytics pipelines must compose graph-parallel and data-
104-
parallel systems, leading to extensive data movement and duplication and a complicated programming
105-
model.
106-
107-
<p style="text-align: center;">
108-
<img src="img/graph_analytics_pipeline.png"
109-
title="Graph Analytics Pipeline"
110-
alt="Graph Analytics Pipeline"
111-
width="50%" />
112-
<!-- Images are downsized intentionally to improve quality on retina displays -->
113-
</p>
114-
115-
The goal of the GraphX project is to unify graph-parallel and data-parallel computation in one
116-
system with a single composable API. The GraphX API enables users to view data both as a graph and
117-
as collections (i.e., RDDs) without data movement or duplication. By incorporating recent advances
118-
in graph-parallel systems, GraphX is able to optimize the execution of graph operations.
119-
120-
<!-- ## GraphX Replaces the Spark Bagel API
121-
122-
Prior to the release of GraphX, graph computation in Spark was expressed using Bagel, an
123-
implementation of Pregel. GraphX improves upon Bagel by exposing a richer property graph API, a
124-
more streamlined version of the Pregel abstraction, and system optimizations to improve performance
125-
and reduce memory overhead. While we plan to eventually deprecate Bagel, we will continue to
126-
support the [Bagel API](api/scala/index.html#org.apache.spark.bagel.package) and
127-
[Bagel programming guide](bagel-programming-guide.html). However, we encourage Bagel users to
128-
explore the new GraphX API and comment on issues that may complicate the transition from Bagel.
129-
-->
130-
13169
## Migrating from Spark 1.1
13270

13371
GraphX in Spark {{site.SPARK_VERSION}} contains a few user facing API changes:
@@ -174,7 +112,7 @@ identifiers.
174112
The property graph is parameterized over the vertex (`VD`) and edge (`ED`) types. These
175113
are the types of the objects associated with each vertex and edge respectively.
176114

177-
> GraphX optimizes the representation of vertex and edge types when they are plain old data types
115+
> GraphX optimizes the representation of vertex and edge types when they are primitive data types
178116
> (e.g., int, double, etc...) reducing the in memory footprint by storing them in specialized
179117
> arrays.
180118
@@ -791,14 +729,13 @@ Graphs are inherently recursive data structures as properties of vertices depend
791729
their neighbors which in turn depend on properties of *their* neighbors. As a
792730
consequence many important graph algorithms iteratively recompute the properties of each vertex
793731
until a fixed-point condition is reached. A range of graph-parallel abstractions have been proposed
794-
to express these iterative algorithms. GraphX exposes a Pregel-like operator which is a fusion of
795-
the widely used Pregel and GraphLab abstractions.
732+
to express these iterative algorithms. GraphX exposes a variant of the Pregel API.
796733

797734
At a high level the Pregel operator in GraphX is a bulk-synchronous parallel messaging abstraction
798735
*constrained to the topology of the graph*. The Pregel operator executes in a series of super steps
799736
in which vertices receive the *sum* of their inbound messages from the previous super step, compute
800737
a new value for the vertex property, and then send messages to neighboring vertices in the next
801-
super step. Unlike Pregel and instead more like GraphLab messages are computed in parallel as a
738+
super step. Unlike Pregel, messages are computed in parallel as a
802739
function of the edge triplet and the message computation has access to both the source and
803740
destination vertex attributes. Vertices that do not receive a message are skipped within a super
804741
step. The Pregel operators terminates iteration and returns the final graph when there are no

0 commit comments

Comments
 (0)