"422 Client Error: Unprocessable Entity" when accessing Pan-STARRS DR2 via MAST astroquery

[fkiwy]

Since last week, I get below error when accessing Pan-STARRS DR2 mean table via following astroquery from the MAST package:

from astroquery.mast import Catalogs

table = Catalogs.query_region(coords, radius=radius, catalog='Panstarrs', data_release='dr2', table='mean',
                              nStackDetections=[('gte', 2)], sort_by=[('asc', 'distance')])

    table = Catalogs.query_region(coords, radius=radius, catalog='Panstarrs', data_release='dr2', table='mean',
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\utils\class_or_instance.py", line 25, in f
    return self.fn(obj, *args, **kwds)
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\utils\process_asyncs.py", line 26, in newmethod
    response = getattr(self, async_method_name)(*args, **kwargs)
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\utils\class_or_instance.py", line 25, in f
    return self.fn(obj, *args, **kwds)
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\mast\collections.py", line 164, in query_region_async
    return self._current_connection.service_request_async(service, params, pagesize=pagesize, page=page)
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\utils\class_or_instance.py", line 25, in f
    return self.fn(obj, *args, **kwds)
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\mast\services.py", line 293, in service_request_async
    response = self._request('POST', request_url, data=catalogs_request, headers=headers, use_json=use_json)
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\astroquery\mast\services.py", line 194, in _request
    response.raise_for_status()
  File "C:\Users\Anaconda3\envs\test-env\lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
  requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://catalogs.mast.stsci.edu/api/v0.1/panstarrs/dr2/mean.json

EDIT: astroquery.mast.Catalogs.query_region seems to produce this error regardless of the specified catalog. All my publicly available tools using this API no longer work.

[bsipocz]

Cc @jaymedina

[Andrew-Swan]

I have been having similar problems - code that has worked for several months is now broken due to this issue. I've discovered that, at least for me, specifying which columns you want to retrieve will cause the HTTP error, but letting it retrieve all columns will work.

from astropy.coordinates import SkyCoord
import astropy.units as u
from astroquery.mast import Catalogs
coords = SkyCoord(ra=100u.deg, dec=25u.deg)
radius = 15*u.arcsec
#This works:
data = Catalogs.query_region(coordinates=coords, radius=radius, catalog="Panstarrs", table="mean")
#This fails:
data = Catalogs.query_region(coordinates=coords, radius=radius, catalog="Panstarrs", table="mean", columns=['objID'])
Traceback (most recent call last):

File "C:\Users\miste\AppData\Local\Temp\ipykernel_2340\2051075302.py", line 1, in
data = Catalogs.query_region(coordinates=coords, radius=radius, catalog="Panstarrs", table="mean", columns=['objID'])

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\utils\class_or_instance.py", line 25, in f
return self.fn(obj, *args, **kwds)

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\utils\process_asyncs.py", line 26, in newmethod
response = getattr(self, async_method_name)(*args, **kwargs)

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\utils\class_or_instance.py", line 25, in f
return self.fn(obj, *args, **kwds)

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\mast\collections.py", line 164, in query_region_async
return self._current_connection.service_request_async(service, params, pagesize=pagesize, page=page)

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\utils\class_or_instance.py", line 25, in f
return self.fn(obj, *args, **kwds)

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\mast\services.py", line 293, in service_request_async
response = self._request('POST', request_url, data=catalogs_request, headers=headers, use_json=use_json)

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\astroquery\mast\services.py", line 194, in _request
response.raise_for_status()

File "C:\Users\miste\Miniconda3\envs\ZTF\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)

HTTPError: 422 Client Error: Unprocessable Entity for url: https://catalogs.mast.stsci.edu/api/v0.1/panstarrs/dr2/mean.json

[fkiwy]

@Andrew-Swan Thanks for sharing this simple workaraound!

My idea was to query the catalog directly via URL and have the same functionality as with astroquery.mast.Catalogs:

        def search_panstarrs(ra, dec, radius):
            url = 'https://catalogs.mast.stsci.edu/api/v0.1/panstarrs/dr2/mean.votable'
            params = {
                'ra': ra,
                'dec': dec,
                'radius': radius.to(u.deg).value,
                'nStackDetections.gte': 2,
                'columns': ['raMean', 'decMean', 'gMeanPSFMag', 'rMeanPSFMag', 'iMeanPSFMag', 'zMeanPSFMag', 'yMeanPSFMag', 'distance'],
                'sort_by': ['distance']
            }
            response = requests.get(url, params=params)
            table = Table.read(BytesIO(response.content), format='votable')
            if len(table) > 0:
                table = table.filled(np.nan)
                return table
            else:
                return None

However, workarounds do not solve the actual problem!