[GraphSage]: Is the size always the same between number of layers in SAGE and len(size) in NeighborSampler?

[udothemath]

There are three SAGEConv examples provided by pytorch geometric. I am wondering is the size of number of neighbors in dataloader always the same as number of SAGEConv layer?

For example 1: reddit, num_neighbors=[25, 10] in Neighborloader, and there are two SAGEConv in SAGE(torch.nn.Module) class.

Similar setting for example 2 and 3.

I can understand the SAGEConv concept of sampling from the node's neighbor, but I don't understand how this idea is implemented in neural network.

In graph, each circle means a node.

In neural network, each circle means a neuron.

How to put these two together? Is there any good reference/visualization to explain the concept? Thanks for your help.

--
For example 2: ogbn_products_sage., sizes=[15, 10, 5] in NeighborSampler, and there are three SAGEConv in SAGE model (num_layers=3).

For example 3: graph_sage_unsup, sizes=[10, 10] in NeighborSampler, and num_layers=2 in SAGE model.

[rusty1s]

You are right. The number of hops (len(num_neighbors)) and the number of layers should match. In your figure, the squared gray boxes denote the application of a GNN layer, and for each round of sampling, we apply a different GNN layer.

A node in the computation graph actually refers to a set of neuron since each node is described by a set of features. For a GNN with input, hidden and output channels equals to 1, a circle describes both a node and a single neuron. Hope this clarifies your issues.

[imethanlee]

Ah, this clarifies all my doubts. Thanks a lot :)

[imethanlee]

Well actually there is one more question. I am using NeighborLoader instead of NeighborSampler now, how can I access the node id within each batch?

Perhaps I can make my question more specific. If I use NeighborLoader to form my subgraph_loader, how can I do efficient inference (by sampling only 1-hop neighbor and update all node hidden representation layer-by-layer as you said in your last reply)? In NeighborSampler we have n_id to access directly but in NeighborLoader I fail to figure out where to access n_id.

[rusty1s]

You can add it as an attribute to data, e.g.: data.n_id = torch.arange(data.num_nodes). That makes it available in the mini-batches of NeighborLoader as well.

[jylink]

I know that the length of num_neighbors and the number of layers in SAGEConv should match, but I can't relate them to the algorithm in the GraphSAGE paper.

For the reddit example, each batch contains 1024 sampled nodes (noted as b3) and their 1-hop (b2) and 2-hop neighbors (b1), and then b3&b2&b1 are passed to the two SAGEConv layers, is that right?

But according to the paper, shouldn't we pass b3&b2&b1 to the first SAGEConv layer and b3&b2 to the second SAGEConv layer?

Also, what is the length of NeighborLoader? How does it determine the number of batches per iteration?

[rusty1s]

Good question. You are right that one usually does not need to pass in the full set of nodes to each layer. We do this for simplicity: We do not need to manage different adjacency matrices across layers, do not need to handle bipartite message passing, and only need to slice the node feature matrix once at the end of the iteration. This simplicity comes with a small increase in runtime since we are actually computing unused node embeddings.

The length of the loader is given by len(seed_nodes) / batch_size.