Categories over a base ring category

[nthiery]

Currently, to construct a category of algebras, we need to specify the
base ring:

    sage: Algebras(QQ)

A side effect is that the construction of e.g. GF(p)['x'] for a
lot of p's in number theoretic calculations triggers the
construction of many parallel hierarchies of categories.

This is a waste because in most situations only the category of the
base ring is relevant. In particular, the generic code provided to
parents/elements/... only depends on the latter.

The purpose of this ticket is to allow for only specifying the
category of the base ring, as in:

    sage: Algebras(Fields())

with

    sage: Algebras(QQ)

constructing a subcategory of the former. Then go trough the code,
decide in each case whether we want to use Algebras(QQ) or
Algebras(Fields()), and update the doctests accordingly. In most
case, the latter idiom will be preferable, unless we need some
operation on the category itself.

This would fix the regression noted on #15792 and made worse by
#10963.

Further features for this ticket or follow-up tickets would be to:

• make Modules(...) into a covariant functorial
  constructions. This would give a proper framework for the feature
  Modules(QQ) -> VectorSpaces(QQ) instead of having to
  tackle it with a special hack.

• implement a PolynomialRings(...) covariant functorial
  construction.

Depends on #10963
Depends on #16275

CC: @sagetrac-sage-combinat @vbraun @simon-king-jena @darijgr

Component: categories

Author: Nicolas M. Thiéry

Branch/Commit: 15658bd

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/15801

[simon-king-jena]

comment:1

As commented on #10963, I see some questions to be addressed.

If I understand correctly, we sometimes do want to specify the base ring/field, and Volker says that this should be by a "mixin" category (lets call it FixedBaseRing(...)) that simply is joined with the non-specific category, say, Algebras(Fields()).

First complication: We not only have base rings, but (for bimodules) left and right base rings. Should this be reflected in the "FixedBaseBla" categories as well?

And another complication: We want to have categories expressing the fact that a field (not just a ring) is acting on the objects, and then want to compute the join. Is there an easy programmatic way to let an error be raised when someone tries to compute the join "Algebras(Fields()) & FixedBaseRing(ZZ)"?

[nthiery]

comment:2

Replying to @simon-king-jena:

So, the strategy would be this: The current "Category over base ring" should accept as input no a ring, but a subcategory of the category of rings.

Or more precisely, it would accept both kinds of input.

A polynomial ring should be initialised in the category of "commutative algebras over Category of quotient fields" (say), and should of course have its own reference to the base ring (say, the rational field).

Yup. And later in the category of "polynomial rings over a quotient field."

Since the parent and element classes only depend on the category of the base ring (by the current implementation), a category refinement would probably not be needed.

Yes.

But if needed, it would amount to take the join with the category "FixedBaseField(QQ), perhaps with super-categories "FixedBaseRing(QQ)".

Yes, though I would write it as taking the join with Algebras(QQ).

Additional technical problem: We not only have base rings, but (for bimodules) left and right base rings. Should this be reflected in the "FixedBaseBla" categories as well?

We will want to accept both:

    Bimodules(Fields(), Fields())

and

    Bimodules(QQ, QQ)

In a first step, I would not bother about mixed calls likes Bimodules(QQ, Fields)/

And another complication: We want to have categories expressing the fact that a field (not just a ring) is acting on the objects, and then want to compute the join. Is there an easy programmatic way to let an error be raised when someone tries to compute the join "Algebras(Fields()) & FixedBaseRing(ZZ)"?

I don't see a trivial way to do it at this point. It certainly would
be a nice feature to bark upon such constructions, but not having it
is not really worst than the current situation where we have:

sage: Algebras(QQ) & Algebras(GF(3))
Join of Category of algebras over Finite Field of size 3 and Category of algebras over Rational Field

So this issue should not prevent us to proceed even if we don't find a good way out.

[simon-king-jena]

comment:3

OK, that's the other possibility: Rather than having a separate mix-in category FixedBaseRing(...), we would allow both Algebras(Fields()) and Algebras(QQ).

I have already done some experiments in that direction. We certainly want Algebras(R.category()) among the super-categories of Algebras(R). But if C is a category, do we want that Algebras(D) is among the super-categories of Algebras(C), for all D in C.super_categories()?

When I tried it yesterday, Nicolas' bounded C3 algorithm barked at me. Reason (if I recall correctly, I can't use my laptop right now): We want that Algebras(QQ) and Algebras(QQ.category() have the same parent and element classes, hence, _make_named_class_key returns the same, and thus they also share the cmp-key used in the C3 algorithm. But the C3 algorithm expects that the items on the given lists (here: the list of all super categories) have distinct keys.

One possible approach to solve that problem: The strict super-categories of a category with fixed base ring should all be categories with unspecified base rings. Hence, Algebras(R) should not have the super-categories Rings() and Modules(R), but Algebras(R.category()).

[nthiery]

comment:4

Replying to @simon-king-jena:

OK, that's the other possibility: Rather than having a separate mix-in category FixedBaseRing(...), we would allow both Algebras(Fields()) and Algebras(QQ).

I have already done some experiments in that direction. We certainly want Algebras(R.category()) among the super-categories of Algebras(R). But if C is a category, do we want that Algebras(D) is among the super-categories of Algebras(C), for all D in C.super_categories()?

When I tried it yesterday, Nicolas' bounded C3 algorithm barked at me. Reason (if I recall correctly, I can't use my laptop right now): We want that Algebras(QQ) and Algebras(QQ.category() have the same parent and element classes, hence, _make_named_class_key returns the same, and thus they also share the cmp-key used in the C3 algorithm. But the C3 algorithm expects that the items on the given lists (here: the list of all super categories) have distinct keys.

One possible approach to solve that problem: The strict super-categories of a category with fixed base ring should all be categories with unspecified base rings. Hence, Algebras(R) should not have the super-categories Rings() and Modules(R), but Algebras(R.category()).

This sounds reasonable. A variant would be, for a category over base
ring C, to set C(R).parent_class explicitly to
C(R.category()).parent_class, bypassing the
_make_named_class_key business since it won't really be needed
anymore anyway (and similarly for the element class and so on).

I'd say that we certainly want Algebras(R) to be a subcategory
of Modules(R), as tested by is_subcategory. But we could
imagine doing like for joins, and not put Modules(R) in
Algebras(R).<all_>super_categories().

In fact, maybe Algebras(R) should really be a join category:
that of Algebras(Fields()) & Modules(R).

Here Modules(R) would play a role similar to the category
with axiom C().A() associated to the category C defining
the axiom A. We would not have the option to define the analogue
of a category with axioms for a subcategory, but at this point I
don't foresee a use for it either.

Cheers,
Nicolas

[simon-king-jena]

Dependencies: 10963

[simon-king-jena]

comment:5

We want to build on top of #10963, right?

Replying to @nthiery:

One possible approach to solve that problem: The strict super-categories of a category with fixed base ring should all be categories with unspecified base rings. Hence, Algebras(R) should not have the super-categories Rings() and Modules(R), but Algebras(R.category()).

This sounds reasonable. A variant would be, for a category over base
ring C, to set C(R).parent_class explicitly to
C(R.category()).parent_class, bypassing the
_make_named_class_key business since it won't really be needed
anymore anyway (and similarly for the element class and so on).

I think _make_named_class_key would still be make sense: We need to distinguish the element and parent classes of Algebras(Fields()) from Algebras(Rings()), but probably not from Algebras(QuotientFields()). And in particular (when following your suggestion below) we would still want that Modules(GF(5)) and Modules(GF(7)) have the same parent and element classes.

I'd say that we certainly want Algebras(R) to be a subcategory
of Modules(R), as tested by is_subcategory. But we could
imagine doing like for joins, and not put Modules(R) in
Algebras(R).<all_>super_categories().

Yes, but then one needs a category FixedBase(R) to be joined with! And I think that's not so nice, as pointed out in comment:1. Or so I thought. But:

In fact, maybe Algebras(R) should really be a join category:
that of Algebras(Fields()) & Modules(R).

Hmmm. Yes, in that case one wouldn't need an ugly FixedBase(R) category...

Here Modules(R) would play a role similar to the category
with axiom C().A() associated to the category C defining
the axiom A. We would not have the option to define the analogue
of a category with axioms for a subcategory, but at this point I
don't foresee a use for it either.

Well, that's what I commented on at #10963: I was playing with the idea of an axiom WithFixedBase, accepting an argument, but Volker has cut the argument short ;-).

[simon-king-jena]

Changed dependencies from 10963 to #10963

[nthiery]

comment:7

Replying to @simon-king-jena:

We want to build on top of #10963, right?

Definitely.

Replying to @nthiery:
I think _make_named_class_key would still be make sense: We need to distinguish the element and parent classes of Algebras(Fields()) from Algebras(Rings()), but probably not from Algebras(QuotientFields()).

That would be for free if we make Algebras a functorial construction:
if a category, say QuotientFields(), says nothing about algebras
over itself, then Algebras(QuotientFields()) will actually
return Algebras(Fields()).

And in particular (when following your suggestion below) we would still want that Modules(GF(5)) and Modules(GF(7)) have the same parent and element classes.

Right, if we go the join way, we will need to handle the base case of
Modules and have a make_named_class_key mechanism there.

[nbruin]

comment:8

Just a little note. In [#10963 comment:261] it was noted that base categories are referenced strongly by a global cache, and hence are prone to cause "memory leaks". This was rebutted there by noting that the number of categories created is uniformly bounded, so the memory use is manageable. This is clearly not true for these parametrized "categories with fixed base".

The work on this ticket will improve the situation, because categories parametrized by elements from an infinite set will be less often created, but it's worthwhile to try and think of what to do if they do get created. Perhaps we should really try and avoid them completely? Certainly the infrastructure requirements to support them is radically different, because such parametrized categories need to be deallocatable (I'm impressed that Firefox's spell checker accepts that word and not Firefox).

[nbruin]

comment:9

A simple test that detects some of the problems #10963 presently causes:

while True: k=random_matrix(Rationals(),30,20).echelon_form()

On 6.2beta4 this happily runs with constant memory footprint and about constant time per iteration.

With #10963 applied, it gradually eats memory. The reason is that "integers mod p" and their categories pile up in memory. The iterations also take about twice as long and they slow down over time (as one would expect with memory use increasing).

(see #13925 for why the choice of algorithm might not be so sensible, but it's a useful test for our purposes here)

[nthiery]

comment:10

Hi!

I started to work on this. We can now do:

    sage: Algebras(Fields())
    Category of algebras over Category of fields

(TODO: would be nicer as Category of algebras over fields)

and by default, polynomial rings are created in such categories:

    sage: QQ['x'].category()
    Join of Category of euclidean domains and Category of commutative algebras over Category of quotient fields

All tests pass in the sage/rings/polynomial.

Here are the new timing for Simon's benchmark about creating lots of
polynomial rings:

With Sage 5.0.1:

sage: %time test()
CPU times: user 7.38 s, sys: 0.08 s, total: 7.46 s
Wall time: 7.48 s

With develop (6.2.beta7):

sage: %time test()
CPU times: user 9.31 s, sys: 125 ms, total: 9.44 s
Wall time: 9.47 s

With #10963:

sage: %time test()
CPU times: user 17.6 s, sys: 288 ms, total: 17.9 s
Wall time: 17.8 s

With #10963 and this ticket:

sage: def test():
....:     for p in prime_range(10000):
....:         P = GF(p)['t','x','z']
sage: %time test()
CPU times: user 2.99 s, sys: 127 ms, total: 3.11 s
Wall time: 3.03 s

Sounds like the time-performance issue for polynomials is indeed fixed for polynomials :-)

Now, on to matrices!