Add the jiter package #1327

FCare · 2025-01-25T09:31:48Z

LICENSE might not be only in the src directory but might be in a subdirectory, as Jiter source which contains LICENSE in src/crates/jitter directory

Closes Install of openai is failing with some issues related to jiter #1227

FCare · 2025-01-25T10:15:37Z

Adding the jiter package

mhsmith · 2025-01-26T19:58:42Z

Many licenses require a copy of the license to be included with all copies of the software, so we must be certain we use the correct license. It's reasonable to assume that a LICENSE file at the top level applies to the entire package, but a file in a subdirectory might only apply to one component. So it would be legally risky to accept this change.

I'd still be happy to accept this PR as a contribution of a recipe for jiter. All you need to do is:

Remove the license autodetection change.
Specify the license path manually with the license_file setting. See meta-schema.yaml for documentation.
Complete the test script.
Run the test using the instructions in pypi/README.md.
Post a comment saying which Python versions and ABIs you've tested.

FCare · 2025-01-27T08:19:48Z

Hi, thanks for your answer, let me clean this. As you may know, pedantic version 2 is used by langgraph and google-generativeai.

It is a hard blocking point to use chaquopy for those recent and attractive technologies.

I will update the PR and continue to try to fix that pedantic stuff.

This reverts commit bfe2644.

FCare · 2025-01-27T14:37:20Z

I removed the license patch and move it in the package yaml file.
I tried to add some tests using the jiter apckage as example and tried to followed the documentation. But the test application does not compile, no idea why.
The resuls logs are these one:

> Task :app:generateDebugPythonRequirements
Chaquopy: Installing for x86_64
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-13.1, file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/
ERROR: Exception:
Traceback (most recent call last):
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
Exception:

  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 335, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/resolution/legacy/resolver.py", line 313, in _get_abstract_dist_for
    self._populate_link(req)
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/resolution/legacy/resolver.py", line 279, in _populate_link
    req.link = self.finder.find_requirement(req, upgrade)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/package_finder.py", line 917, in find_requirement
    best_candidate_result = self.find_best_candidate(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/package_finder.py", line 900, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/package_finder.py", line 844, in find_all_candidates
    package_links = self.process_project_url(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/package_finder.py", line 809, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/collector.py", line 622, in fetch_page
    return _get_html_page(location, session=self.session)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/collector.py", line 457, in _get_html_page
    resp = _get_html_response(url, session=session)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/index/collector.py", line 151, in _get_html_response
    resp = session.get(
           ^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/network/session.py", line 421, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_vendor/requests/sessions.py", line 683, in send
    r.content
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_vendor/requests/models.py", line 829, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_vendor/requests/models.py", line 762, in generate
    chunk = self.raw.read(chunk_size)
            ^^^^^^^^^^^^^
AttributeError: 'FileNotFoundError' object has no attribute 'read'
Chaquopy: Exit status 2

> Task :app:generateDebugPythonRequirements FAILED
Execution failed for task ':app:generateDebugPythonRequirements'.
> Process 'command '/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/bin/python'' finished with non-zero exit value 1

FCare · 2025-01-27T15:10:56Z

I added a verbose option to pip install. I thoink it is my integration with PIP to use local whl file which is not working .Do you have any advice?


Fetching project page and analyzing links: https://chaquo.com/pypi-13.1/jiter/
Getting page https://chaquo.com/pypi-13.1/jiter/
Found index url https://chaquo.com/pypi-13.1
Looking up "https://chaquo.com/pypi-13.1/jiter/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): chaquo.com:443
https://chaquo.com:443 "GET /pypi-13.1/jiter/ HTTP/1.1" 404 273
Status code 404 not in (200, 203, 300, 301)
Could not fetch URL https://chaquo.com/pypi-13.1/jiter/: 404 Client Error: Not Found for url: https://chaquo.com/pypi-13.1/jiter/ - skipping
Fetching project page and analyzing links: file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/jiter/
 file: URL is directory, getting file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/jiter/index.html
Getting page file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/jiter/index.html
Found index url file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/
ERROR: Exception:
Removed build tracker: '/tmp/pip-req-tracker-tlxqyizb'
Traceback (most recent call last):
  File "/Work/Inno/sandbox/pr/chaquopy/server/pypi/pkgtest/app/build/python/env/debug/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
             ^^^^^^^^^^^^^^^^^^^^^^^

Integration is:

// To test packages, edit the following line to list their names, separated by
// spaces. Each name must be a subdirectory of PACKAGES_DIR.
def PACKAGES = "jiter"
addPackages(delegate, PACKAGES.trim().split(/\s+/).toList())

    python {
        version "3.11"

        pip {
            // After a package is tested, remember to move it to the public repository!
            options "--extra-index-url", "file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/"
        }
        pyc {
            src false  // Include test source code in stack traces.
        }
        extractPackages "astropy"
    }

    ndk {
        abiFilters "x86_64"
     }

FCare · 2025-01-27T15:24:29Z

If I copy the whl in the chaquopy cache, overriding package in cache and adding the option "--force-reinstall", then the test is passing:

So issue is mainly about the local.whl usage with the gradle file

FCare · 2025-01-27T15:48:55Z

I added the test suite to cover the full set of jiter tests.
It is all passing.

I tested on android emulator, so x86_64, python 3.11.

How do I test on more possibilities?

mhsmith · 2025-02-02T19:14:33Z

options "--extra-index-url", "file:///Work/Inno/sandbox/pr/chaquopy/server/pypi/dist/"

I'm not sure if our current version of pip will accept the file:// syntax here. Try just giving a plain path.

I tested on android emulator, so x86_64, python 3.11.
How do I test on more possibilities?

I'll test the other possibilities myself once the PR is ready. But it needs a few more changes first:

You've included a lot of unnecessary (sometimes whitespace-only) changes. Please reduce it to just the changes that are needed by this package.
Temporary package-specific changes in the pkgtest app should also not be committed.
The test file is too big. We don't want to run the package's entire test suite, because that makes it too slow to test multiple packages before each Chaquopy release. All we need is a single test of the package's basic functionality, like in the other recipes.
The test file uses pytest, but the test harness is based on unittest. This will be easier to fix once the file has been reduced to a single test.

mhsmith · 2025-02-02T19:36:03Z

As you may know, pedantic version 2 is used by langgraph and google-generativeai.

It is a hard blocking point to use chaquopy for those recent and attractive technologies.

As discussed in #1328, it doesn't look like pydantic actually depends on the Python jiter package at all.

Francois Caron added 2 commits January 25, 2025 10:29

Search recursively for LICENSE file - Fix Jiter package build

bfe2644

Add jiter 0.8.0 package

013769b

mhsmith changed the title ~~Search recursively for LICENSE file~~ Add the jiter package Jan 26, 2025

mhsmith marked this pull request as draft January 26, 2025 19:59

mhsmith mentioned this pull request Jan 26, 2025

building pydantic-core with modified jiter #1326

Closed

Francois Caron added 4 commits January 27, 2025 09:22

Revert "Search recursively for LICENSE file - Fix Jiter package build"

02c3f87

This reverts commit bfe2644.

Use yaml to specify license location

ca77a1c

wip

c07763b

Trigger ci by hand

3f6ca79

USe the full jiter test suite

fe6924b

mhsmith mentioned this pull request Feb 2, 2025

Add pydantic core #1328

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the jiter package #1327

Add the jiter package #1327

FCare commented Jan 25, 2025 •

edited by mhsmith

Loading

FCare commented Jan 25, 2025

mhsmith commented Jan 26, 2025 •

edited

Loading

FCare commented Jan 27, 2025

FCare commented Jan 27, 2025 •

edited by mhsmith

Loading

FCare commented Jan 27, 2025 •

edited

Loading

FCare commented Jan 27, 2025

FCare commented Jan 27, 2025

mhsmith commented Feb 2, 2025

mhsmith commented Feb 2, 2025 •

edited

Loading

Add the jiter package #1327

Are you sure you want to change the base?

Add the jiter package #1327

Conversation

FCare commented Jan 25, 2025 • edited by mhsmith Loading

FCare commented Jan 25, 2025

mhsmith commented Jan 26, 2025 • edited Loading

FCare commented Jan 27, 2025

FCare commented Jan 27, 2025 • edited by mhsmith Loading

FCare commented Jan 27, 2025 • edited Loading

FCare commented Jan 27, 2025

FCare commented Jan 27, 2025

mhsmith commented Feb 2, 2025

mhsmith commented Feb 2, 2025 • edited Loading

FCare commented Jan 25, 2025 •

edited by mhsmith

Loading

mhsmith commented Jan 26, 2025 •

edited

Loading

FCare commented Jan 27, 2025 •

edited by mhsmith

Loading

FCare commented Jan 27, 2025 •

edited

Loading

mhsmith commented Feb 2, 2025 •

edited

Loading