Release v0.24.0
We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.24.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), Learning Engine(GLE), and Deployment.
We highlight the following improvements included in this release:
1. Enhancements for GIE
Key Features:
- Support the
sample()
step of Gremlin to randomly pick up a given number of traversers from a Gremlin traversal. Additionally,coin()
step can now follow any step in a Gremlin traversal, instead of justV()
andE()
. For example,
g.V().sample(10) // Sample 10 traversers from `g.V()`
g.V().out().coin(0.2) // Randomly pick up 20% among all vertices' out neighbors
- Support the
union
step of Gremlin to combine the traversers from multiple traversals. For example,
g.V().union(out(), out().out()) // the results would now be one-hop (out once) and two-hop (out twice) neighbors of all vertices.
- Support the
unfold
step of Gremlin, which is a reversed operator offold
that flattens a collection of traversers into discrete elements. For example,
g.V().fold().unfold() // must obtain `g.V()` again
- Support the
identity()
step of Gremlin, which simply returns the current traversers, and is often used together withunion
step. For example,
g.V().union(identity(), out()) // return `V()` and its one-hop neighbors
- Support
isNull
in theexpr()
for verifying wether a certain property hasNULL
value. For example,
g.V().where(expr("@.age isNull")) // the vertices without the field of `age` will be pruned
But fixes:
- Fix a bug of
count()
step followed byorder()
andlimit()
that can cause thegaia
engine to abort unexpectedly. Now the following Gremlin query can be executed smoothly:
g.V().order().by('id', asc).limit(1).out().count()
2. Enhancements for Deployment
- All GraphScope Images support both
x86
andarm64
platform - This release includes the gsctl command-line utility for building and testing GraphScope. The gsctl tool provides several commands and options to streamline the development and deployment process. You can use the following command to install the graphscope-client package and then use gsctl:
$ pip3 install graphscope-client
$ gsctl --help
3. Other enhancements and bug fixes
- Reduced the size of GRAPE-java shaded jar,reduce shaded jars' total size from 94MB to 58MB.
- Introduce Hiactor-based High-QPS Engine for Flex.
- Refine the schema definition for flex
rt_mutable_graph
- Fix the
Py_None
reference count issue in GLE - Fix the bug that GLE client only connects with a single server in distributed training
- Update the GLE-related tutorials
Docker Image
# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.24.0
# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.24.0
# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.24.0
# Executor component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.24.0
# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.24.0
# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.24.0
# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest
What's Changed
- Fixes the docs generation for tagged releases by @sighingnow in #2970
- Fixes compliation error for non-builtin frames on MacOS by @sighingnow in #2972
- Fix memory leak in groot and add trace logs by @siyuan0322 in #2971
- Refine GC related codes in groot by @siyuan0322 in #2973
- Bump grpc-protobuf from 1.47.0 to 1.53.0 in /analytical_engine/java by @dependabot in #2977
- Bump up the version of kubernetes-log-export-action to v5 by @dashanji in #2976
- [GIE Compiler] Set Cypher Query Timeout from System Configurations by @shirly121 in #2975
- Support cypher query in python by @andydiwenzhu in #2952
- [Coordinator] Fix issue #2734: unifying timeout parameter in coordinators by @TaoLbr1993 in #2915
- [Bug Fix] Fix bugs of time counter in gremlin service by @shirly121 in #2906
- [GIE Compiler] Support Procedure Call in Cypher Queries by @shirly121 in #2927
- Pre-release 0.24.0 by @lidongze0629 in #2986
- Make GraphPlanner able to accept file as input by @zhanglei1949 in #2980
- [Flex] Some modification on Flex code by @zhanglei1949 in #2990
- [GIE] Introduce Aggregate.First proto definition by @zhanglei1949 in #2991
- Update GLE tutorials and bump GLE version by @LiSu in #2983
- [GIE] Fix unbalanced data routing in DefaultRouter in GIE by @BingqingLyu in #2994
- Use arrow::LargeStringArray in trivial_tensor_t to store data larger than 2GB by @acezen in #2999
- Revised groot-client and add release phrase by @siyuan0322 in #3000
- Retry download when checksum failed in groot data-loading process by @siyuan0322 in #3006
- [Flex] Upgrade
libgrape-lite
version by @zhanglei1949 in #3004 - Add deprecated decorator and apply it on gremlin by @siyuan0322 in #3007
- [GIE Doc] Refine Docs for Cypher by @shirly121 in #2995
- [GIE] Add LSQB QuerySet into Benchmark Tool by @BingqingLyu in #2997
- [GIE Compiler] Make Config path Configurable from User Given Parameters by @shirly121 in #2982
- unit test for pegasus timeout by @lnfjpt in #2964
- [GIE Test] Add Timeout CI Test in Python by @shirly121 in #2989
- Fix figure display issues in GIE LDBC tutorial by @MeloYang05 in #3009
- update libgrape-lite's commit id in Flex dockerfile by @zhanglei1949 in #3016
- [BugFix] Fix bug when process pk scan of multiple labels by @BingqingLyu in #3013
- [Flex] Update flex dockerfile by @zhanglei1949 in #3020
- [Flex] Introducing Hiactor-based HQPS Engine into flex by @zhanglei1949 in #2981
- Upgrade to latest vineyard (v0.16.1) by @sighingnow in #3019
- Enable perfect hash support in GAE by @sighingnow in #2959
- Fixes the Python version detection in MacOS CI by @sighingnow in #3022
- Update README.md by @yecol in #3027
- [Flex] Introducing Hiactor-based HQPS Engine into flex by @zhanglei1949 in #3024
- Fixed jdk_1.8 version for data-loading-tool by @lidongze0629 in #3031
- unify gae, python and groot proto into proto directory by @siyuan0322 in #3023
- Implement grin interface for flex by @liulx20 in #3010
- ci: Add check for conventional commit and docs by @yecol in #3033
- [GIE Compiler] Introduce LDBC CI Tests for Cypher Query by @shirly121 in #2984
- Update README.md to reflect GraphScope Flex by @wenyuanyu in #3035
- doc: Update README.md by adding latest news section by @yecol in #3036
- revise ldbc apps by @siyuan0322 in #3034
- [Bug Fix] Fix Incremental Query Id Issues in Multiple Frontend by @shirly121 in #3026
- Fixes the implementation of adjlist in flatten fragment by @sighingnow in #3042
- Fixes resolve protobuf library on MacOS. by @sighingnow in #3043
- Propogate error message about argument mismatch to Python client by @sighingnow in #3044
- Fixes MacOS CI for UDFs by correcting the library path using install_name_tool by @sighingnow in #3045
- [BugFix] Fix networkx add edge segment fault bug by @acezen in #3032
- Add the performance & tunning page for GAE by @sighingnow in #3046
- Integrate sdk-common into common for simplisity by @siyuan0322 in #3050
- fix: make LDBC metadata consistent in experiment store with other storages by @BingqingLyu in #3048
- ci: add conventional checks for PR title and docs by @yecol in #3054
- fix: Fixes error in docs generation by removing the submodule by @sighingnow in #3056
- fix: Fix Incorrect Hyperlinks in GIE Doc
supported_gremlin_steps.md
by @shirly121 in #3057 - feat(analytical): Add the Graph.consolidate_columns() interface in Python client by @sighingnow in #3060
- feat(analytical): support use int32_t as vid_t to save memory for small graphs by @sighingnow in #3063
- refactor(analytical): Remove
nativeLibLoader
in GRAPE-jdk by @zhanglei1949 in #3065 - fix(analytical): fixes wrong result when flatten from projected arrow fragment by @hanjf12 in #3066
- ci: address the failure of packaging graphscope wheels on MacOS by @sighingnow in #3068
- Add config to enable multiple frontend support in groot deployment by @siyuan0322 in #3073
- ci: Add GitHub workflow to build graphscope-dev:wheel image by @lidongze0629 in #3072
- docs: add loading graphs doc by @siyuan0322 in #3075
- ci: fixes workflow of building GraphScope wheel image by @lidongze0629 in #3083
- ci(flex): Add flex in pr-check's scopes by @zhanglei1949 in #3084
- feat(analytical): Enable cython app on local vertex map graph by @siyuan0322 in #3081
- chore: Refine the logic to probe GRAPHSCOPE_HOME by @siyuan0322 in #3082
- refactor(analytical): Reduce shaded jar size. by @zhanglei1949 in #3076
- fix(interactive): fix unexpected results for some group().by() queries by @BingqingLyu in #3099
- docs: add a note about the pip version requirements by @sighingnow in #3105
- fix(k8s): fixes the dockerfile to download kubectl using correct arch by @sighingnow in #3104
- fix(interactive): fix a bug of count operator by @lnfjpt in #3067
- fix(interactive): Fix Examples in GIE Doc by @shirly121 in #3108
- feat(interactive): Refine the GrootClient for concurrent and async write. by @siyuan0322 in #3113
- fix(learning): bump GLE version to include the latest bug fix by @LiSu in #3115
- refactor(flex): Refine the schema definition of
rt_mutable_graph
by @zhanglei1949 in #3079 - chore: bump up vineyard version to v0.16.4 by @sighingnow in #3106
- chore: bump up gremlinpython to 3.7.0 by @siyuan0322 in #3119
- refactor(interactive): Join operator in pegasus will repartition stream before join by @lnfjpt in #3118
- Update README.md to fix a spell error by @xiezheng-XD in #3123
- ci: fixes doc generation failure on CI by @sighingnow in #3125
- feat(interactive): Implement GraphScope Interactive by @zhanglei1949 in #3030
- fix(interactive): throw unsupported error when query
PathExpand()
within subtask by @BingqingLyu in #3133 - chore: allow user pass variables for substitutions by @siyuan0322 in #3135
- fix(flex): Fix hqps engine's support for some simple
MATCH
queries by @zhanglei1949 in #3086 - fix(interactive): fix the docs of interactive engine by @longbinlai in #3131
- feat(interactive): support isNull expression evaluation in GIE by @BingqingLyu in #3128
- feat(interactive): Support Identity step in GIE by @Louyk14 in #3111
- fix(interactive): fix bug in unfold case
g.V().fold().as("a").unfold()
in GIE Runtime by @BingqingLyu in #3130 - feat(interactive): Support
Sample()
andCoin()
steps in GIE by @BingqingLyu in #3091 - refactor: Upgrade the build phase of image in CI with gsctl by @FLYLX in #3142
- fix(interactive): Support Multiple Matches in IR Core by @BingqingLyu in #3134
- refactor(flex): Small modifications to the graph schema to fit the GIE compiler. by @zhanglei1949 in #3150
- refactor(interactive): Initiate Compiler Service with Yaml Configuration by @shirly121 in #3141
- ci: Support to build dev images(both x86 and aarch64) in CI workflow by @lidongze0629 in #3153
- fix(interactive): fixes some unsafe rust code that cannot be compiled with recent stable rust compilers by @sighingnow in #3160
- refactor: Upgrade gsctl and modify files associated with gsctl by @FLYLX in #3149
- ci: fixes failure in building GraphScope wheel package by @lidongze0629 in #3164
- feat(interactive): Support Optional Match and Where Not(subquery) in Compiler by @shirly121 in #3088
- refactor(interactive): refactor the implementation of
PathExpand
by @BingqingLyu in #3159 - Differentiating between arm64 and x86 wheel's name by @lidongze0629 in #3165
New Contributors
- @hanjf12 made their first contribution in #3066
- @xiezheng-XD made their first contribution in #3123
- @Louyk14 made their first contribution in #3111
- @FLYLX made their first contribution in #3142
Full Changelog: v0.23.0...v0.24.0