Releases: mars-project/mars
v0.7.0a3
This is the release notes of v0.7.0a3. See here for the complete list of solved issues and merged PRs.
New Features
- DataFrame
- Implements
head()
on groupby objects #1849
- Implements
- Learn
- Implements
mars.learn.preprocessing.{MinMaxScaler, minmax_scale}
(#1844)
- Implements
Enhancements
- Improve Proxima
recall_by_id
computation method (#1805, thanks @rg070836rg!) - Revise to/from vineyard, for Tensor and DataFrame. (#1790)
- Add memory estimation for
read_parquet
as well asread_csv
(#1811) - Support using compound agg function in lambda (#1810)
- Add
incremental_index
argument toreset_index
which by default is False (#1823) - Refine kubedl cluster-api. (#1827, thanks @SimonCqk!)
- Enhancements for Mars on kubedl (#1848)
- Support
to_pandas
in a batch way for DataFrame and Series (#1853) - Support specifying memory limit scale in kubernetes (#1856)
- Set marsjob worker cache by
memoryTuningPolicy
. (#1857, thanks @SimonCqk!)
Bug fixes
v0.7.0a2
This is the release notes of v0.7.0a2. See here for the complete list of solved issues and merged PRs.
New Features
- Tensor
- Support
missing
argument fortensor.tosparse()
andfill_value
argument forsparse_tensor.todense()
(#1797)
- Support
- DataFrame
Bug fixes
- Spawn serialization of executable graphs (#1769)
- Fix getitem on DataFrames with unknown index (#1772)
- Fix reading partitioned parquet files in HDFS (#1782)
- Fix creating Mars Series from empty pandas Series (#1787)
- Support md.concat on DataFrame and Series (#1798)
- Fix bug that explicit execute may be required for to_parquet and XGB predict (#1794)
- Fix TypeError when timeout argument is absent when starting Mars cluster in YARN (#1803, thanks @smartguo!)
Documentation
- Fill docs for
apply
andtransform
(#1764)
Tests
- Create different test workflows & fix accessor docs (#1799)
v0.6.1
This is the release notes of v0.6.1. See here for the complete list of solved issues and merged PRs.
New Features
- Tensor
- Support
missing
argument fortensor.tosparse()
andfill_value
argument forsparse_tensor.todense()
(#1802)
- Support
- DataFrame
Bug fixes
- Spawn serialization of executable graphs (#1770)
- Fix getitem on DataFrames with unknown index (#1778)
- Fix reading partitioned parquet files in HDFS (#1783)
- Fix creating Mars Series from empty pandas Series (#1788)
- Fix bug that explicit execute may be required for to_parquet and XGB predict (#1800)
- Support md.concat on DataFrame and Series (#1801)
- Fix TypeError when timeout argument is absent when starting Mars cluster in YARN (#1804, thanks @smartguo!)
Documentation
- Fill docs for apply and transform (#1767)
Tests
- Create different test workflows & fix accessor docs (#1804)
v0.7.0a1
This is the release notes of v0.7.0a1. See here for the complete list of solved issues and merged PRs.
Changes that break compatibility
- Aggregations and Groupby with aggregations have been rewritten in v0.6.0, older client may raise error when connecting to cluster with new version installed.
Highlights
- Statsmodels as well as joblib are preliminarily supported.
New Features
- DataFrame
- Tensor
- Add rebalance method for tensors (#1731)
- Learn
Bug fixes
- Fix
md.read_csv
whennames
andusecols
specified (#1737) - Make PSRS chunks more balanced (#1742)
- Support string dtype for tensor reductions (#1745)
- Fix xgboost and lightgbm on DataFrames (#1750)
- Fix repeated execution of same code in distributed mode (#1749)
- Support setting scalar which is a tensor for DataFrame (#1755)
v0.6.0
This is the release notes of v0.6.0. See here for the complete list of solved issues and merged PRs.
This release note only covers the difference from v0.6.0rc1; for all highlights and changes, please refer to the release notes of the pre-releases:
alpha1
alpha2
alpha3
beta1
beta2
rc1
Changes that break compatibility
- Aggregations and Groupby with aggregations have been rewritten in v0.6.0, older client may raise error when connecting to cluster with new version installed.
New Features
- DataFrame
Enhancements
- Unify
groupby.agg()
using ReductionCompiler (#1739)
Bug fixes
- Fix
md.read_csv
whennames
andusecols
specified (#1738) - Support string dtype for tensor reductions & balance PSRS chunks (#1746)
- Fix XGBoost and LightGBM on DataFrames (#1751)
- Fix repeated execution of same code in distributed mode (#1753)
- Support setting scalar which is a tensor for DataFrame (#1758)
v0.6.0rc1
This is the release notes of v0.6.0rc1. See here for the complete list of solved issues and merged PRs.
New Features
- DataFrame
- Implements
{DataFrame,Series}.explode
(#1714)
- Implements
- Learn
- Support predicting on local LGBM models (#1716)
Enhancements
- Add configuration page on Mars Web (#1697)
- Add shared limit option for Mars worker (#1702)
- Remount the shm directory in
entrypoint.sh
(#1700) - Add pure-dependent option for operands (#1706)
- Remove
prepare_inputs
property on operands (#1709) - Use ReductionCompiler to support function aggregation in
mars.dataframe.reduction
(#1705) - Write into and read from merged files when data sizes are small (#1708)
- Refactor builder and searcher of Proxima (#1710, thanks @rg070836rg!)
Bug fixes
- Fix mars not working on ray cluster (#1712, thanks @fyrestone!)
- Fix inferring dtype for
series.map
(#1722) - Fix sort functions of DataFrames on CUDA (#1723)
v0.5.5
This is the release notes of v0.5.5. See here for the complete list of solved issues and merged PRs.
New Features
- DataFrame
- Implements
{DataFrame,Series}.explode
(#1715)
- Implements
Enhancements
- Add configuration page on Mars Web (#1701)
- Remount
/dev/shm
directory inentrypoint.sh
in Kubernetes and limit plasma size to avoid SIGBUS (#1703) - Add pure-dependent option for operands (#1707)
Bug fixes
v0.6.0b2
This is the release notes of v0.6.0b2. See here for the complete list of solved issues and merged PRs.
New Features
- Learn
- Support recall computation for proxima (#1657, thanks @rg070836rg!)
Enhancements
- Support
inplace
parameter inreset_index
method (#1662) - Add a threshold for
DataFrame.head
optimization (#1673)
Bug fixes
- Check unknown shape chunks in tile of
md.concat
(#1655) - Create Fetch operands given output types (#1666)
- Fix hang for rerun
DataFrame.groupby
in distributed mode (#1667) - Modify
df.copy()
so that it generates the identical key (#1671) - Fix IndexError when binary op on Series whose type is datetime (#1675)
- Mount /dev/shm on host to pods when starting Mars workers in Kubernetes (#1677)
- Fix Series reduction that output type consistent for map and combine phase (#1685)
- Fix wrong dtypes of DataFrame
setitem
chunks (#1690) - Add timeout for
SharedHolderActor
creation (#1684) - Fix assigning operands with expected workers (#1689)
v0.5.4
This is the release notes of v0.5.4. See here for the complete list of solved issues and merged PRs.
Enhancements
- Support
inplace
parameter inreset_index
method (#1663) - Add a threshold for
DataFrame.head
optimization (#1679)
Bug fixes
- Check unknown shape chunks in tile of
md.concat
(#1656) - Fix hang for rerun
DataFrame.groupby
in distributed mode (#1669) - Create Fetch operands given output types (#1668)
- Modify
df.copy()
so that it generates the identical key (#1678) - Fix
IndexError
when binary op on Series whose type is datetime (#1680) - Mount /dev/shm on host to pods when starting Mars workers in Kubernetes (#1681)
- Fix DataFrame reduction that output type consistent for map and combine phase (#1686)
- Fix wrong dtypes of DataFrame
setitem
chunks (#1691) - Fix assigning operands with expected workers (#1693)
- Add timeout for SharedHolderActor creation (#1692)
v0.6.0b1
This is the release notes of v0.6.0b1. See here for the complete list of solved issues and merged PRs.
New Features
- DataFrame
- Add
DataFrame.to_parquet
support (#1652)
- Add
Enhancements
- Optimize memory usage for brute-force algorithm in
NearestNeighbors
(#1640) - Structural adjustment for proxima (#1624, thanks @rg070836rg!)
Bug fixes
- Fix the wrong dtypes of
DataFrameSetitem
's inputs (#1623) - Fix issue that
output_type
does not take effect fordf.apply
(#1626) - Fix registration for
DataFrameSetLabel
operand (#1631) - Fix issue that serialization of transpose failed when input has unknown shape (#1632)
- Eliminate TimeoutError when there are running nodes (#1637)
- Fix PSRS error when chunks has fewer rows than partition number (#1642)
- Add
flush
method to_LogWrapper
(#1646) - Fix
md.concat
which may occupy huge amount of memory on client when all of DataFrames own largeRangeIndex
(#1649)