postgres problem psycopg2.InterfaceError: connection already closed

[marco-foscato]

On MacOS
aiida_core 1.0.0.b3
postgres 11.2 installed via Homebrew
python 3.7.3

Error from pytest on aiida-diff after the number of connection (see jamesnunn/pgtest#14):

self = 
autocommit = False

    def _set_autocommit(self, autocommit):
        with self.wrap_database_errors:
>           self.connection.autocommit = autocommit
E           psycopg2.InterfaceError: connection already closed

../aiida_virtualenv/lib/python3.7/site-packages/django/db/backends/postgresql/base.py:236: InterfaceError

[ltalirz]

@marco-foscato sorry for the long wait - could you check whether adding determine_setup=False as an argument to the Postgres constructor in the following line fixes the issue?
https://github.com/aiidateam/aiida_core/blob/develop/aiida/manage/fixtures.py#L160

You need to keep the fix we added for pgtest with the number of connections (which I'll now update).

[marco-foscato]

Tired, but it raises the ValueError: Could not connect to postgres. from aiida_core/aiida/manage/external/pgsu.py:111. With determine_setup=False the connection_mode of Postgress is never changed from its default DISCONNECTED value (see https://github.com/aiidateam/aiida_core/blob/develop/aiida/manage/external/pgsu.py#L78).

You probably know this already, but without the determine_setup=False the database cluster is created and the aiida-diff test runs at least to some extent. Then, while the aiida_diff/tests/test_calculations.py is hanging, I can still get the psql promt and see the aiida_db database with some aiida nodes:

aiida_db=# SELECT * FROM db_dbnode;
 id |                 uuid                 |                node_type                 | label | description |             ctime             |             mtime             | nodeversion | public | dbcomputer_id | user_id |      process_type
----+--------------------------------------+------------------------------------------+-------+-------------+-------------------------------+-------------------------------+-------------+--------+---------------+---------+-------------------------
  1 | b746dce5-5713-4b5d-a4b4-68653ccfa6e0 | data.code.Code.                          | diff  |             | 2019-05-28 10:04:31.518223+02 | 2019-05-28 10:04:31.521425+02 |           1 | f      |             1 |       1 |
  2 | 2b6849e6-6516-4e91-8aed-7b48e7668ce7 | data.diff.DiffParameters.                |       |             | 2019-05-28 10:04:31.556153+02 | 2019-05-28 10:04:31.748153+02 |           1 | f      |               |       1 |
  3 | a8714fac-d8a1-4042-b6b6-02a4667ee368 | data.singlefile.SinglefileData.          |       |             | 2019-05-28 10:04:31.556416+02 | 2019-05-28 10:04:31.758851+02 |           1 | f      |               |       1 |
  4 | b3df5b14-166a-4bf0-9070-9e9415f797b0 | data.singlefile.SinglefileData.          |       |             | 2019-05-28 10:04:31.557821+02 | 2019-05-28 10:04:31.769294+02 |           1 | f      |               |       1 |
  6 | 2a422a6b-1edd-4164-b130-d678edda72f4 | data.remote.RemoteData.                  |       |             | 2019-05-28 10:04:34.421252+02 | 2019-05-28 10:04:34.458573+02 |           1 | f      |             1 |       1 |
  5 | 3867082f-f9ae-4175-adc6-3ede4ab17998 | process.calculation.calcjob.CalcJobNode. |       |             | 2019-05-28 10:04:31.638074+02 | 2019-05-28 10:04:36.871604+02 |          21 | f      |             1 |       1 | aiida.calculations:diff

[ltalirz]

@marco-foscato Very sorry for keeping this issue open for so long - unfortunately, we cannot easily test it from our side.
PR #3892 has now moved the "pgsu" module out into a separate package that is tested on a wide range of operating systems and postgres setups. It also included a couple of fixes, so perhaps this issue is resolved as well?
Let us know!

[sphuber]

Closing due to inactivity. Feel free to reopen if the problem persists and needs to be addressed.