Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert numpy.floating values in meta.json #13644

Closed
wants to merge 44 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
82fc2ec
Bump version to 3.7.5 (#13493)
svlandeg May 15, 2024
a6d0fc3
Remove typing-extensions from requirements (#13516)
svlandeg May 31, 2024
f78e5ce
Disable extra CI
honnibal Jun 21, 2024
8cda27a
Add case study [ci skip]
ines Jun 26, 2024
a8accc3
Use cibuildwheel to build wheels (#13603)
honnibal Aug 20, 2024
319e025
Set version to 3.7.6
honnibal Aug 20, 2024
55db9c2
Added gd language folder (#13570)
markliberko Sep 9, 2024
acbf2a4
Add Kurdish Kurmanji language (#13561)
cikay Sep 9, 2024
608f65c
add Tibetan (#13510)
ykyogoku Sep 9, 2024
1b8d560
Support 'memory zones' for user memory management (#13621)
honnibal Sep 9, 2024
b65491b
Set version to v3.8.0.dev0
honnibal Sep 9, 2024
59ac7e6
Format
honnibal Sep 9, 2024
a019315
Fix memory zones
honnibal Sep 9, 2024
4cc3ebe
Format
honnibal Sep 9, 2024
b18cc94
Delete unnecessary method (#13441)
marinelay Sep 9, 2024
5a7ad55
added gliner-spacy to universe (#13417) [ci skip]
wjbmattingly Sep 10, 2024
54dc4ee
Added: Constituent-Treelib to: universe.json (#13432) [ci skip]
Halvani Sep 10, 2024
0190e66
universe-package-quelquhui (#13514) [ci skip]
thjbdvlt Sep 10, 2024
081e4e3
universe-project-presque (#13515) [ci skip]
thjbdvlt Sep 10, 2024
89c1774
added bagpipes-spacy to universe (#13425) [ci skip]
wjbmattingly Sep 10, 2024
7fbbb20
updated universe for number spacy (#13424) [ci skip]
wjbmattingly Sep 10, 2024
c80dacd
added spacy annoy to universe (#13416) [ci skip]
wjbmattingly Sep 10, 2024
f1a5ff9
added spacy whisper to universe (#13418) [ci skip]
wjbmattingly Sep 10, 2024
30f1f33
Added Date spaCy to universe (#13415) [ci skip]
wjbmattingly Sep 10, 2024
184e508
Update numpy pin
honnibal Sep 11, 2024
c068e1d
Fix dependencies
honnibal Sep 11, 2024
1869a19
Try enabling macos-14 for arm builds
honnibal Sep 11, 2024
b427597
Set version to v3.8.0
honnibal Sep 11, 2024
69ecb85
Set version to v3.8.1
honnibal Sep 13, 2024
419bfaf
Update cibuildwheel
honnibal Sep 13, 2024
83b4015
Remove aarch
honnibal Sep 13, 2024
a0ce61f
Fix thinc pin
honnibal Sep 13, 2024
3a635d2
Try skipping 686
honnibal Sep 13, 2024
8dcc4b8
Skip running tests on PRs
honnibal Sep 14, 2024
8266031
Merge numpy version update
honnibal Sep 14, 2024
50aa3b5
Merge branch 'master' of https://github.com/explosion/spaCy
honnibal Sep 14, 2024
3c3d750
Set version to v3.7.7
honnibal Sep 14, 2024
e2dc9b7
Format
honnibal Sep 14, 2024
2f1e7ed
Lint
honnibal Sep 14, 2024
a9ed8bb
Replace numpy floats in evaluate and update
honnibal Sep 14, 2024
57cbac7
Fix numpy floating values in meta.json for serialization
honnibal Sep 14, 2024
5dde59a
Format
honnibal Sep 30, 2024
e36a178
Add missing import
honnibal Oct 1, 2024
07dba26
Remove obsolete python versions from tests
honnibal Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions .github/workflows/cibuildwheel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
name: Build

on:
push:
tags:
# ytf did they invent their own syntax that's almost regex?
# ** matches 'zero or more of any character'
- 'release-v[0-9]+.[0-9]+.[0-9]+**'
- 'prerelease-v[0-9]+.[0-9]+.[0-9]+**'
jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-latest, windows-latest, macos-13, macos-14]

steps:
- uses: actions/checkout@v4
# aarch64 (arm) is built via qemu emulation
# QEMU is sadly too slow. We need to wait for public ARM support
#- name: Set up QEMU
# if: runner.os == 'Linux'
# uses: docker/setup-qemu-action@v3
# with:
# platforms: all
- name: Build wheels
uses: pypa/[email protected]
env:
CIBW_ARCHS_LINUX: auto
with:
package-dir: .
output-dir: wheelhouse
config-file: "{package}/pyproject.toml"
- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

build_sdist:
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Build sdist
run: pipx run build --sdist
- uses: actions/upload-artifact@v4
with:
name: cibw-sdist
path: dist/*.tar.gz
create_release:
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
permissions:
contents: write
checks: write
actions: read
issues: read
packages: write
pull-requests: read
repository-projects: read
statuses: read
steps:
- name: Get the tag name and determine if it's a prerelease
id: get_tag_info
run: |
FULL_TAG=${GITHUB_REF#refs/tags/}
if [[ $FULL_TAG == release-* ]]; then
TAG_NAME=${FULL_TAG#release-}
IS_PRERELEASE=false
elif [[ $FULL_TAG == prerelease-* ]]; then
TAG_NAME=${FULL_TAG#prerelease-}
IS_PRERELEASE=true
else
echo "Tag does not match expected patterns" >&2
exit 1
fi
echo "FULL_TAG=$TAG_NAME" >> $GITHUB_ENV
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
echo "IS_PRERELEASE=$IS_PRERELEASE" >> $GITHUB_ENV
- uses: actions/download-artifact@v4
with:
# unpacks all CIBW artifacts into dist/
pattern: cibw-*
path: dist
merge-multiple: true
- name: Create Draft Release
id: create_release
uses: softprops/action-gh-release@v2
if: startsWith(github.ref, 'refs/tags/')
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
name: ${{ env.TAG_NAME }}
draft: true
prerelease: ${{ env.IS_PRERELEASE }}
files: "./dist/*"
File renamed without changes.
29 changes: 29 additions & 0 deletions .github/workflows/publish_pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# The cibuildwheel action triggers on creation of a release, this
# triggers on publication.
# The expected workflow is to create a draft release and let the wheels
# upload, and then hit 'publish', which uploads to PyPi.

on:
release:
types:
- published

jobs:
upload_pypi:
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/spacy
permissions:
id-token: write
contents: read
if: github.event_name == 'release' && github.event.action == 'published'
# or, alternatively, upload to PyPI on every tag starting with 'v' (remove on: release above to use this)
# if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
steps:
- uses: robinraju/release-downloader@v1
with:
tag: ${{ github.event.release.tag_name }}
fileName: '*'
out-file-path: 'dist'
- uses: pypa/gh-action-pypi-publish@release/v1
6 changes: 2 additions & 4 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: tests

on:
push:
tags-ignore:
- '**'
branches-ignore:
- "spacy.io"
- "nightly.spacy.io"
Expand Down Expand Up @@ -59,10 +61,6 @@ jobs:
os: [ubuntu-latest, windows-latest, macos-latest]
python_version: ["3.12"]
include:
- os: windows-latest
python_version: "3.7"
- os: macos-latest
python_version: "3.8"
- os: ubuntu-latest
python_version: "3.9"
- os: windows-latest
Expand Down
59 changes: 56 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,64 @@ requires = [
"cymem>=2.0.2,<2.1.0",
"preshed>=3.0.2,<3.1.0",
"murmurhash>=0.28.0,<1.1.0",
"thinc>=8.2.2,<8.3.0",
"numpy>=1.15.0; python_version < '3.9'",
"numpy>=1.25.0; python_version >= '3.9'",
"thinc>=8.3.0,<8.4.0",
"numpy>=2.0.0,<2.1.0; python_version < '3.9'",
"numpy>=2.0.0,<2.1.0; python_version >= '3.9'",
]
build-backend = "setuptools.build_meta"

[tool.cibuildwheel]
build = "*"
skip = "pp* cp36* cp37* cp38* *-win32 *i686*"
test-skip = ""
free-threaded-support = false

archs = ["native"]

build-frontend = "default"
config-settings = {}
dependency-versions = "pinned"
environment = { PIP_CONSTRAINT = "build-constraints.txt" }

environment-pass = []
build-verbosity = 0

before-all = "curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain stable"
before-build = "pip install -r requirements.txt && python setup.py clean"
repair-wheel-command = ""

test-command = ""
before-test = ""
test-requires = []
test-extras = []

container-engine = "docker"

manylinux-x86_64-image = "manylinux2014"
manylinux-i686-image = "manylinux2014"
manylinux-aarch64-image = "manylinux2014"
manylinux-ppc64le-image = "manylinux2014"
manylinux-s390x-image = "manylinux2014"
manylinux-pypy_x86_64-image = "manylinux2014"
manylinux-pypy_i686-image = "manylinux2014"
manylinux-pypy_aarch64-image = "manylinux2014"

musllinux-x86_64-image = "musllinux_1_2"
musllinux-i686-image = "musllinux_1_2"
musllinux-aarch64-image = "musllinux_1_2"
musllinux-ppc64le-image = "musllinux_1_2"
musllinux-s390x-image = "musllinux_1_2"

[tool.cibuildwheel.linux]
repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel}"

[tool.cibuildwheel.macos]
repair-wheel-command = "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"

[tool.cibuildwheel.windows]

[tool.cibuildwheel.pyodide]


[tool.isort]
profile = "black"
5 changes: 2 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ catalogue>=2.0.6,<2.1.0
typer>=0.3.0,<1.0.0
weasel>=0.1.0,<0.5.0
# Third party dependencies
numpy>=1.15.0; python_version < "3.9"
numpy>=1.19.0; python_version >= "3.9"
numpy>=2.0.0; python_version < "3.9"
numpy>=2.0.0; python_version >= "3.9"
requests>=2.13.0,<3.0.0
tqdm>=4.38.0,<5.0.0
pydantic>=1.7.4,!=1.8,!=1.8.1,<3.0.0
Expand All @@ -22,7 +22,6 @@ langcodes>=3.2.0,<4.0.0
# Official Python utilities
setuptools
packaging>=20.0
typing_extensions>=3.7.4.1,<4.5.0; python_version < "3.8"
# Development dependencies
pre-commit>=2.13.0
cython>=0.25,<3.0
Expand Down
9 changes: 4 additions & 5 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -36,21 +36,21 @@ python_requires = >=3.7
# spaCy v4
setup_requires =
cython>=0.25,<3.0
numpy>=1.15.0; python_version < "3.9"
numpy>=1.19.0; python_version >= "3.9"
numpy>=2.0.0,<2.1.0; python_version < "3.9"
numpy>=2.0.0,<2.1.0; python_version >= "3.9"
# We also need our Cython packages here to compile against
cymem>=2.0.2,<2.1.0
preshed>=3.0.2,<3.1.0
murmurhash>=0.28.0,<1.1.0
thinc>=8.2.2,<8.3.0
thinc>=8.3.0,<8.4.0
install_requires =
# Our libraries
spacy-legacy>=3.0.11,<3.1.0
spacy-loggers>=1.0.0,<2.0.0
murmurhash>=0.28.0,<1.1.0
cymem>=2.0.2,<2.1.0
preshed>=3.0.2,<3.1.0
thinc>=8.2.2,<8.3.0
thinc>=8.3.0,<8.4.0
wasabi>=0.9.1,<1.2.0
srsly>=2.4.3,<3.0.0
catalogue>=2.0.6,<2.1.0
Expand All @@ -66,7 +66,6 @@ install_requires =
# Official Python utilities
setuptools
packaging>=20.0
typing_extensions>=3.7.4.1,<4.5.0; python_version < "3.8"
langcodes>=3.2.0,<4.0.0

[options.entry_points]
Expand Down
2 changes: 1 addition & 1 deletion spacy/about.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# fmt: off
__title__ = "spacy"
__version__ = "3.7.4"
__version__ = "3.7.7"
__download_url__ = "https://github.com/explosion/spacy-models/releases/download"
__compatibility__ = "https://raw.githubusercontent.com/explosion/spacy-models/master/compatibility.json"
16 changes: 16 additions & 0 deletions spacy/lang/bo/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from ...language import BaseDefaults, Language
from .lex_attrs import LEX_ATTRS
from .stop_words import STOP_WORDS


class TibetanDefaults(BaseDefaults):
lex_attr_getters = LEX_ATTRS
stop_words = STOP_WORDS


class Tibetan(Language):
lang = "bo"
Defaults = TibetanDefaults


__all__ = ["Tibetan"]
16 changes: 16 additions & 0 deletions spacy/lang/bo/examples.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""
Example sentences to test spaCy and its language models.

>>> from spacy.lang.bo.examples import sentences
>>> docs = nlp.pipe(sentences)
"""


sentences = [
"དོན་དུ་རྒྱ་མཚོ་བླ་མ་ཞེས་བྱ་ཞིང༌།",
"ཏཱ་ལའི་ཞེས་པ་ནི་སོག་སྐད་ཡིན་པ་དེ་བོད་སྐད་དུ་རྒྱ་མཚོའི་དོན་དུ་འཇུག",
"སོག་པོ་ཨལ་ཐན་རྒྱལ་པོས་རྒྱལ་དབང་བསོད་ནམས་རྒྱ་མཚོར་ཆེ་བསྟོད་ཀྱི་མཚན་གསོལ་བ་ཞིག་ཡིན་ཞིང༌།",
"རྗེས་སུ་རྒྱལ་བ་དགེ་འདུན་གྲུབ་དང༌། དགེ་འདུན་རྒྱ་མཚོ་སོ་སོར་ཡང་ཏཱ་ལའི་བླ་མའི་སྐུ་ཕྲེང་དང་པོ་དང༌།",
"གཉིས་པའི་མཚན་དེ་གསོལ་ཞིང༌།༸རྒྱལ་དབང་སྐུ་ཕྲེང་ལྔ་པས་དགའ་ལྡན་ཕོ་བྲང་གི་སྲིད་དབང་བཙུགས་པ་ནས་ཏཱ་ལའི་བླ་མ་ནི་བོད་ཀྱི་ཆོས་སྲིད་གཉིས་ཀྱི་དབུ་ཁྲིད་དུ་གྱུར་ཞིང་།",
"ད་ལྟའི་བར་ཏཱ་ལའི་བླ་མ་སྐུ་ཕྲེང་བཅུ་བཞི་བྱོན་ཡོད།",
]
65 changes: 65 additions & 0 deletions spacy/lang/bo/lex_attrs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
from ...attrs import LIKE_NUM

# reference 1: https://en.wikipedia.org/wiki/Tibetan_numerals

_num_words = [
"ཀླད་ཀོར་",
"གཅིག་",
"གཉིས་",
"གསུམ་",
"བཞི་",
"ལྔ་",
"དྲུག་",
"བདུན་",
"བརྒྱད་",
"དགུ་",
"བཅུ་",
"བཅུ་གཅིག་",
"བཅུ་གཉིས་",
"བཅུ་གསུམ་",
"བཅུ་བཞི་",
"བཅུ་ལྔ་",
"བཅུ་དྲུག་",
"བཅུ་བདུན་",
"བཅུ་པརྒྱད",
"བཅུ་དགུ་",
"ཉི་ཤུ་",
"སུམ་ཅུ",
"བཞི་བཅུ",
"ལྔ་བཅུ",
"དྲུག་ཅུ",
"བདུན་ཅུ",
"བརྒྱད་ཅུ",
"དགུ་བཅུ",
"བརྒྱ་",
"སྟོང་",
"ཁྲི་",
"ས་ཡ་",
" བྱེ་བ་",
"དུང་ཕྱུར་",
"ཐེར་འབུམ་",
"ཐེར་འབུམ་ཆེན་པོ་",
"ཁྲག་ཁྲིག་",
"ཁྲག་ཁྲིག་ཆེན་པོ་",
]


def like_num(text):
"""
Check if text resembles a number
"""
if text.startswith(("+", "-", "±", "~")):
text = text[1:]
text = text.replace(",", "").replace(".", "")
if text.isdigit():
return True
if text.count("/") == 1:
num, denom = text.split("/")
if num.isdigit() and denom.isdigit():
return True
if text in _num_words:
return True
return False


LEX_ATTRS = {LIKE_NUM: like_num}
Loading
Loading