Fix python version in setup for numba errors

[sky-2002]

Description: As per the issue given here, numba requires python version <3.12 and >=3.8. According to the PEP 440 version specifiers, I have added <3.12 in the setup function in setup.py file.

Note: The installation will still have the issue, but now the setup will have python version requirements so users can change versions accordingly.

[ddkang]

Thanks for putting this together, this would be our first external contribution!

I think since numba is our only 3.12 blocker, we should actually make numba optional and jit the functions on demand, if it's available. Do you think you could do that instead?

[sky-2002]

Hi @ddkang, I could see numba being used only in one function, so I have tried making it optional (checking with import). Let me know if that works.

[ddkang]

I believe you can just define the function and move the shared code out of the try/except block

[sky-2002]

Hi @ddkang , moved shared code outside, let me know if you need any changes.

[ddkang]

I meant you can simply share the function name and call the function outside of the try/except block

[sky-2002]

You mean like this? -

def get_and_update_dists(x: np.ndarray, embeddings: np.ndarray, min_dists: np.ndarray):
  '''
  :param x: embedding of cluster representatives
  :param embeddings: embeddings of all data
  :min_dists: array to record the minimum distance for each embedding to the embedding of cluster representatives
  '''
  try:
    from numba import njit, prange

    @njit(parallel=True)
    def _get_and_update_dists(x: np.ndarray, embeddings: np.ndarray, min_dists: np.ndarray):

      for i in prange(len(embeddings)):
        _get_and_update_dists_shared(x, i, min_dists, embeddings)

  except:
    def _get_and_update_dists(x: np.ndarray, embeddings: np.ndarray, min_dists: np.ndarray):
      for i in range(len(embeddings)):
        _get_and_update_dists_shared(x, i, min_dists, embeddings)
  
  _get_and_update_dists(x, embeddings, min_dists)

[ddkang]

yes

[sky-2002]

Okay, pushed the changes.

[ddkang]

@ttt-77 can you review these changes?

[ttt-77]

I think it's better to move it into function 'get_and_update_dists'.

[sky-2002]

Okay @ttt-77 , moved it.

[ttt-77]

The code logic looks good to me. Could you test it with numba and without numba if possible?

[sky-2002]

The code logic looks good to me. Could you test it with numba and without numba if possible?

Sure. One thing I wanted to ask, as numba is still there in requirements, users would still get the python version related error due to numba installation. Do we need to also make changes to requirements or what?

[ddkang]

Can we make the requirement optional? Like aidb[all] vs aidb[base] or something

[sky-2002]

@ddkang Sorry for a late reply, cant we just add numba[some_name] in requirements instead of just numba. So that it only gets installed when someone does - pip install aidb[some_name]. Will that be good?

[ddkang]

Sure sounds good

[sky-2002]

But I think aidb cannot be installed as pip install aidb as of now. So maybe temporarily, we can go with the method given here, which says that we can specify python version after the package: for example - numba; python_version=='3.11' in the requirements. Should I add this and push?

[sky-2002]

@ddkang Do you also have a slack or discord channel for aidb, I am planning to add a new vectorDB to aidb(written code already), so I may need to ask doubts there.

[ddkang]

Hi @sky-2002 could you email me for the slack invite link? Thank you for helping with the development!

Sounds good regarding the temporary solution.

[ttt-77]

But I think aidb cannot be installed as pip install aidb as of now. So maybe temporarily, we can go with the method given here, which says that we can specify python version after the package: for example - numba; python_version=='3.11' in the requirements. Should I add this and push?

Try pip install ai-db