Reverse relation is lost for abstract models with custom querysets

[leamingrad]

What's wrong

The reverse relation to a concrete subclass of an abstract model that uses a custom queryset is not being populated by django-stubs.

A full set of test cases that illustrate the situation are in a commit here that I added on top of the latest master.

Since an example is worth a thousand words:

from django.db import models

class AbstractLogQuerySet(models.QuerySet):
    pass

class Transaction(models.Model):
    pass

class AbstractLog(models.Model):
    objects = AbstractLogQuerySet.as_manager()
    class Meta:
        abstract=True

class TransactionLog(AbstractLog):
    transaction = models.ForeignKey(Transaction, on_delete=models.CASCADE, related_name="logs")

Transaction().logs
# myapp/models:13: error: "Transaction" has no attribute "logs"

Interestingly, the problem seems to be specific to querysets being used to make managers:

• An abstract model with the default manager works fine
• An abstract model with a custom manager works fine
• An abstract model with a custom manager and queryset fails as well

I'm happy to help with a fix, but to be honest I have no idea where to start.

System information

I've seen this on multiple Python/Django/mypy/django-stubs versions but the test cases are written against:

• python version: 3.9.2
• django version:3.2.4
• mypy version: 0.910
• django-stubs version: HEAD

(Apologies if this is a duplicate - I looked and there seem to be similar issues but not this exact one)

[sobolevn]

I'm happy to help with a fix, but to be honest I have no idea where to start.

Great to hear! I would be happy to help.

1. Start with a failing test somewhere here: https://github.com/typeddjango/django-stubs/tree/master/tests/typecheck/db/models Your example would be a great starting point
2. Then, look at the plugin itself, the execution path you are interested in is:

• get_base_class_hook https://github.com/typeddjango/django-stubs/blob/master/mypy_django_plugin/main.py#L289 ->
• transform_model_class

  django-stubs/mypy_django_plugin/main.py

  Line 30 in 159a0e4

  def transform_model_class(ctx: ClassDefContext, django_context: DjangoContext) -> None:

  ->
• process_model_class

  django-stubs/mypy_django_plugin/transformers/models.py

  Line 363 in 159a0e4

  def process_model_class(ctx: ClassDefContext, django_context: DjangoContext) -> None:

  ->
• AddRelatedManagers

  django-stubs/mypy_django_plugin/transformers/models.py

  Line 237 in 159a0e4

  class AddRelatedManagers(ModelClassInitializer):

Ask any questions you have! 👍

[leamingrad]

I've managed to make a bit of progress on this, but would appreciate any help as I am a bit stuck.

From what I can tell, the root of the problem is when the AddRelatedManagers transformer is dealing with the TransactionLog class in the example above, it does not have an objects property.

This should be being set by the AddManagers transformer, but it fails to do so because it is never able to find the type info for the manager class when the from_queryset method is involved (I have switched my examples to use that as it generally seems to be more supported than .from_manager).

I have managed to make some progress in the commit here by making sure that we always look for generated managers. This makes things work when we change the example above to:

from django.db import models

class AbstractLogQuerySet(models.QuerySet):
    pass

class Transaction(models.Model):
    pass

# Declare the queryset manager outside the model, and use from_queryset
AbstractLogQSManager = models.Manager.from_queryset(AbstractLogQuerySet)

class AbstractLog(models.Model):
    objects = AbstractLogQSManager()
    class Meta:
        abstract=True

class TransactionLog(AbstractLog):
    transaction = models.ForeignKey(Transaction, on_delete=models.CASCADE, related_name="logs")

Transaction().logs
# No output - so everything is fine

However, it still does not work when the manager declaration is inline in the AbstractLog class. When I move the declaration of objects inside TransactionLog, everything works as expected, but I guess that is because the AddManagers transformation does not need to do anything as the objects property is declared explicitly?

I guess my confusion boils down to: "is there a way to make sure that create_new_manager_class_from_from_queryset_method" is called when a manager is declared from a queryset within an abstract class rather than globally?

[sobolevn]

You try to defer() if objects's type are not yet analyzed. This might help.

[RJPercival]

@leamingrad, all of your test cases now pass on master. Could you please re-test to confirm the issue you encountered is now fixed?

[leamingrad]

Thanks for this - just reran my tests locally that were reproducing the issue and I can confirm that this is now fixed 🎉

[sobolevn]

As someone living in Leningrad, I appreciate your username @leamingrad 😆