Rewrite why annotation scope is needed (#4)

* Rewrite why annotation scope is needed

* Minor copyediting

Author: jimbaker

peps/pep-0750.rst

@@ -414,53 +414,69 @@ except that all expressions are always implicitly wrapped with a ``lambda``:
     lexical scope, including local and global variables. Any valid Python expression
     can be used, including function and method calls.
 
-However, this PEP further proposes to extend the usual lexical scoping of a
-lambda-wrapped expression in the interpolation, as seen with class definitions,
-to that supported by `annotation scopes
-<https://docs.python.org/3/reference/executionmodel.html#annotation-scopes>`_,
-so as to minimize developer surprise.
+However, there's one additional nuance to consider, `function scope
+<https://docs.python.org/3/reference/executionmodel.html#resolution-of-names>`_
+versus `annotation scope
+<https://docs.python.org/3/reference/executionmodel.html#annotation-scopes>`_.
+Consider this somewhat contrived example to configure captions:
 
-Let's look at why using annotation scope is important with a somewhat contrived
-example where a developer sets up how figures are rendered in a caption, using a
-``html`` tag:
+.. code-block:: python
+
+    class CaptionConfig:
+        tag = 'b'
+        figure = f'<{tag}>Figure</{tag}>'
+
+Let's now attempt to rewrite the above example to use tag strings:
 
 .. code-block:: python
 
-    class SomeConfig:
+    class CaptionConfig:
         tag = 'b'
         figure = html'<{tag}>Figure</{tag}>'
 
-Unless the ``html`` function fully evaluates all interpolations before the class
-is constructed, a subsequent interpolation will fail with ``NameError: name
-'tag' is not defined``. This means that the name ``tag`` is no longer available
-in the usual lexical scope for lambdas. Contrast that scoping with what is seen
-in using a f-string:
+Unfortunately, this rewrite doesn't work if using the usual lambda wrapping to
+implement interpolations, namely ``lambda: tag``. When the interpolations are
+evaluated by the tag function, it will result in ``NameError: name 'tag' is not
+defined``. The root cause of this name error is that ``lambda: tag`` uses function scope,
+and it's therefore not able to use the class definition where ``tag`` is
+defined.
+
+Desugaring how the tag string could be evaluated will result in the same
+``NameError`` even using f-strings; the lambda wrapping here also uses function
+scoping:
 
 .. code-block:: python
 
-    class SomeConfig:
+    class CaptionConfig:
         tag = 'b'
-        figure = f'<{tag}>Figure</{tag}>'
+        figure = f'<{(lambda: tag)()}>Figure</{(lambda: tag)()}>'
 
-The class variable ``figure`` here does evaluate correctly with respect to
-lexical scope, if at the risk of an HTML injection attack. The reason it
-evaluates correctly is that the evaluation is always immediate. First the
-expression for ``tag`` is evaluated; then for ``figure``; and finally both
-settings are passed into the dynamic construction of the class ``SomeConfig``.
+For tag strings, getting such a ``NameError`` would be surprising. It would also
+be a rough edge in using tag strings in this specific case of working with class
+variables. After all, tag strings are supposed to support a superset of the
+capabilities of f-strings.
 
-Because tag strings are supposed to work like f-strings, but with more
-capabilities, it's necessary to support annotation scope for the lambda-wrapped
-expressions in interpolations.
+The solution is to use annotation scope for tag string interpolations. While the
+name "annotation scope" suggests it's only about annotations, it solves this
+problem by lexically resolving names in the class definition, such as ``tag``,
+unlike function scope.
 
-But why is annotation scope used, given that this PEP is not at all about
-annotations?
+.. note::
+
+    The use of annotation scope means it's not possible to fully desugar
+    interpolations into Python code. Instead it's as if one is writing
+    ``interpolation_lambda: tag``, not ``lambda: tag``, where a hypothetical
+    ``interpolation_lambda`` keyword variant uses annotation scope instead of
+    the standard function scope.
+
+    This is more or less how the reference implementation implements this
+    concept (but without creating a new keyword of course).
 
-*Annotation* scope (as of :pep:`649` and :pep:`695`) provides the necessary
-scoping semantics for names used in class definitions, thereby avoiding the
-``NameError`` above. In addition, the implementation of these two PEPs provide a
-somewhat similar deferred execution model for annotations. However, this PEP and
-its reference implementation only use the support for annotation scope; it's up
-to the tag function to evaluate any interpolations.
+This PEP and its reference implementation therefore use the support for
+annotation scope. Note that this usage is a separable part from the
+implementation of :pep:`649` and :pep:`695` which provides a somewhat similar
+deferred execution model for annotations. Instead it's up to the tag function to
+evaluate any interpolations.
 
 With annotation scope in place, lambda-wrapped expressions in interpolations
 then provide the usual lexical scoping seen with f-strings. So there's no need
@@ -520,7 +536,7 @@ the following, at the cost of losing some type specificity:
     def mytag(*args: str | tuple) -> Any:
         ...
 
-A user might write a tag string as a split string:
+A user might write a tag string as follows:
 
 .. code-block:: python
 
@@ -568,7 +584,14 @@ This is equivalent to:
 
 .. code-block:: python
 
-    mytag(DecodedConcrete(r'Hi, '), InterpolationConcrete(lambda: name, 'name', 's', 'format_spec'), DecodedConcrete(r'!'))
+    mytag(DecodedConcrete(r'Hi, '), InterpolationConcrete(lambda: name, 'name',
+    's', 'format_spec'), DecodedConcrete(r'!'))
+
+.. note::
+
+    To keep it simple, this and subsequent desugaring omits an important scoping
+    aspect in how names in interpolation expressions are resolved, specifically
+    when defining classes. See `Interpolation Expression Evaluation`_.
 
 No Empty Decoded String
 -----------------------