Add SGFormer model and example

[xnuohz]

Issue

• From Graph Transformer Enhancement #9751
• Adds an example for training SGFormer model on ogbn-arxiv dataset.

Usage

python examples/sgformer.py

Highlight

Results

Dataset	Original Paper	PyG
ogbn-arxiv	72.63%	72.40%

[codecov]

Codecov Report

Attention: Patch coverage is 82.73381% with 24 lines in your changes missing coverage. Please review.

Project coverage is 86.31%. Comparing base (1ab3993) to head (b414941).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
torch_geometric/nn/models/sgformer.py	80.80%	19 Missing ⚠️
torch_geometric/nn/attention/sgformer.py	86.84%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9904      +/-   ##
==========================================
+ Coverage   85.84%   86.31%   +0.46%     
==========================================
  Files         490      492       +2     
  Lines       32537    32696     +159     
==========================================
+ Hits        27932    28220     +288     
+ Misses       4605     4476     -129     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[puririshi98]

please put the Model from example into torch_geometric.nn.models
then also put a unit test under test/nn/models.
then please integrate the model into examples/ogbn_train.py.
do this with "--gnn-choice" arg parse option, with choices ["sage, gat, polynormer_graph_transformer"]

please share the log of all 3 and then change the default to the best

[xnuohz]

examples/ogbn_train.py

model	valid acc	test acc
GAT	70.99%	69.93%
GraphSage	71.47%	70.47%
SGFormer	72.96%	71.18%

ogbn_graphsage.log
ogbn_sgformer.log
ogbn_gat.log

[puririshi98]

awesome! can you add the same setup to examples/ogbn_train_cugraph.py and examples/multi_gpu/ogbn_train_cugraph.py and run the same tests

[puririshi98]

btw i ran papers100 got 65% test acc (waiting to hear back from a colleague what the latest results for gat and sage are to save myself some re-doing effort)

[xnuohz]

I was stuck in installing the cugraph_pyg env with conda. I am installing nv-pyg docker, after that, will test it under cugraph env

[xnuohz]

examples/ogbn_train_cugraph.py

model	valid acc	test acc	avg training time	avg inference time
GCN	71.82%	70.64%	13.3698	15.3689
GAT	71.56%	70.23%	13.5808	15.4540
GraphSage	70.63%	67.08%	13.0129	15.2490
SGFormer	72.76%	71.48%	9.6251	6.4098

ogbn_cugraph_gat.log
ogbn_cugraph_gcn.log
ogbn_cugraph_sage.log
ogbn_cugraph_sgformer.log

examples/multi_gpu/ogbn_train_cugraph.py

model	valid acc	test acc	avg training time	avg inference time
GCN	71.25%	70.21%	4.4272	4.9305
GAT	70.93%	69.35%	4.8759	4.7953
GraphSage	71.11%	69.15%	3.9777	4.4666
SGFormer	70.22%	67.90%	9.7087	6.8623

ogbn_cugraph_multi_gat.log
ogbn_cugraph_multi_gcn.log
ogbn_cugraph_multi_sage.log
ogbn_cugraph_multi_sgformer.log