transparent inline leak underlying opaque types

[soronpo]

The transparent inline macro that exists within an opaque source can leak the underlying opaque type.
Note that this does not occur if the macro is not transparent, or if the inline is not a macro.
Additionally, we require another pass through a transparent leak definition to expose this leak (in this example it is implemented with given ... with, but that could be a transparent inline that does the same thing).

Compiler version

v3.1.0-RC1

Minimized code

Minimized project at: https://github.com/soronpo/dottybug/tree/transparent_opaque_leak

MyOpaque.scala

import scala.quoted.*
opaque type MyOpaque = Int
object MyOpaque:
  val one: MyOpaque = 1
  transparent inline def apply(): MyOpaque = ${ applyMacro }
  private def applyMacro(using Quotes): Expr[MyOpaque] =
    import quotes.reflect.*
    '{ one }

Leak.scala

trait Leak[T]:
  type Out
given [T]: Leak[T] with
  type Out = T
extension [T](t: T)(using l: Leak[T]) def leak: l.Out = ???

val x = MyOpaque().leak
val shouldWork = summon[x.type <:< MyOpaque]

Output

[error] 8 |val shouldWork = summon[x.type <:< MyOpaque]
[error]   |                                            ^
[error]   |               Cannot prove that (x : given_Leak_T[Int]#Out) <:< MyOpaque.
[error] one error found

Expectation

No error.

[odersky]

Too many elements at play here. Somebody more familiar with macros should take this on.

[nicolasstucki]

Minimized to

object Bar:
  opaque type MyOpaque = Int
  object MyOpaque:
    transparent inline def apply(): MyOpaque = 1

object Foo:
  import Bar._
  val shouldFail: Int = MyOpaque()
  val shouldWork: MyOpaque = MyOpaque()

[nicolasstucki]

@odersky it seems that when we have a transparent inline we are refining the return type to the inlined expression. I thought we did not allow transparent inlines to be defined in a context where we have opaque types.

[smarter]

This is intentional, using transparent means you want the result type to depend on the computed type on the right-hand-side which in this case is 1, though we actually end up returning the intersection MyOpaque & 1 to stay type correct, see d4e8f5d and associated discussion in #12815 (comment).

[nicolasstucki]

Then it seems we are missing the MyOpaque & part in the generated tree. We get

    val shouldFail: Int = 1
    val shouldWork: Bar.MyOpaque = 1

where we should probably get

    val shouldFail: Int = 1.asInstanceOf[Bar.MyOpaque & 1]
    val shouldWork: Bar.MyOpaque = 1.asInstanceOf[Bar.MyOpaque & 1]

[nicolasstucki]

In this case, we don't execute this branch d4e8f5d#diff-03b5fb619563f54fbceece0e370239c21e3908fada0a395dd2c4c3e5280bd3b6R1003

[nicolasstucki]

A similar case reported in #14305

class Example {
  opaque type Index = Array[Int]
  transparent inline def index(using index: Index): Index = index
}

object Usage {
  def run(example: Example)(using example.Index): Unit = {
    example.index.length
  }
}

[prolativ]

I bumped against this issue myself as well. Now I'm wondering what should be considered a proper use case for a transparent inline def returning an opaque type. E.g. I have some simplified code:

Opaque.scala:

package foo

trait Impl1
trait Impl2

object OpaqueScope:
  opaque type Wrapper = Impl1
  transparent inline def wrapper: Wrapper = new Impl1 {}

Leak.scala:

package bar

import foo.{Impl1, OpaqueScope}

val x: Impl1 = OpaqueScope.wrapper

My original intention was that wrapper should return Wrapper with a type refinement (whose exact type was computed in a macro, which is not that important here however) so that's why wrapper would have to be transparent to preserve the refinement. At the same time as Wrapper is an opaque type I wouldn't expect its implementation details to leak out. This might cause problems e.g. if Leak.scala was some code in a downstream project and the implementation of Wrapper was changed to opaque type Wrapper = Impl2. Then the code in Leak.scala would stop compiling.

Wouldn't it then make sense to assume that when a usage of a transparent inline def returning an opaque type is inlined outside of the definition scope of that opaque type then the compiler should treat the opaque type and its RHS as unrelated and only allow narrowing the declared return type of the def by returning a subtype of the opaque type (e.g. Wrapper with a refinement in my example)?

[markehammons]

I have hit this issue as well:

https://scastie.scala-lang.org/j5yMFWwwQoStHQ2LIEtDUw

object Test:
  opaque type NegativeInt = Int 
  
  object NegativeInt:
    transparent inline def apply(i: Int) = 
      if i < 4 then applyC(i) else applyR(i)

    private def applyC(i: Int): NegativeInt = i
    private def applyR(i: Int): Option[NegativeInt] = Some(i)

import Test.NegativeInt

val a: NegativeInt = NegativeInt(3) //fails 

I'm trying to use the following to create an apply method that, if data can be checked at compiletime, runs compiletime checks with potential error messages, otherwise pushes the check to runtime and returns Option[A] instead.

[TomasMikula]

I am also hitting what seems to be a variation of this problem. Adding my minimized use case.

Compiler version

3.4.1-RC1

Minimized code

macros.scala

import scala.quoted.*

// Cannot specify a more precise return type than `Any`,
// as it is only determined by the return type of `f`, which itself
// may be generated by a transparent inline.
transparent inline def appliedTo7(inline f: Int => Any): Any =
  ${ appliedTo7Impl('f) }

private def appliedTo7Impl(f: Expr[Int => Any])(using Quotes): Expr[Any] =
  '{ $f(7) }

main.scala

object Module:
  opaque type Foo = Int

  transparent inline def mkFoo(i: Int): Int => Foo =
    (j: Int) => (i + j): Foo

import Module.*

val x: Foo = 
  appliedTo7(mkFoo(8))

Output

[error] -- [E007] Type Mismatch Error: /Users/tomas/tmp/transparent-inline-opaque/main.scala:10:12 
[error] 10 |  appliedTo7(mkFoo(8))
[error]    |  ^^^^^^^^^^^^^^^^^^^^
[error]    |  Found:    Int
[error]    |  Required: Module.Foo
[error]    |----------------------------------------------------------------------------
[error]    | Explanation (enabled by `-explain`)
[error]    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]    |
[error]    | Tree: {
[error]    |   val $proxy2: Module.type{type Foo = Int} =
[error]    |     Module.$asInstanceOf[Module.type{type Foo = Int}]
[error]    |   val Module$_this: ($proxy2 : Module.type{type Foo = Int}) = $proxy2
[error]    |   15:Module$_this.Foo
[error]    | }
[error]    | I tried to show that
[error]    |   Int
[error]    | conforms to
[error]    |   Module.Foo
[error]    | but none of the attempts shown below succeeded:
[error]    |
[error]    |   ==> Int  <:  Module.Foo  = false
[error]    |
[error]    | The tests were made under the empty constraint
[error]     ----------------------------------------------------------------------------
[error] one error found
[error] (Compile / compileIncremental) Compilation failed

Expectation

Should compile, as I'd expect the explicit type annotation in (i + j): Foo to be preserved by inlining.

[BalmungSan]

We hit this same issue today during a conversation in the Discord server.

Trying to solve the original problem, we reached this code: https://scastie.scala-lang.org/BalmungSan/DzlQjmPBTuSZ08PIXqtbmw/46
But, as you can see, that was leaking the underlying type.
In this case, however, it was easy to "workaround" the issue by just moving the extension to another object: https://scastie.scala-lang.org/BalmungSan/DzlQjmPBTuSZ08PIXqtbmw/52 This means the compiler no longer has access to the information behind the opaque; Which I think somewhat proves that it is possible for the compiler to both refine the return type but not leave the opaque boundaries.

I am actually surprised this was originally reported more than 3 years ago. And, I don't think this is working as expected.
Yes, the idea of transparent is to be able to refine the return type based on the inputs, but an opaque should not be leaked outside of its scope.

[soronpo]

Maybe @Gedochao can prioritize this since several different people have hit this time and again.

[som-snytt]

Fully qualified result works.

package i20302:

  opaque type Opaque = Int
  transparent inline def op: Opaque = 123
  transparent inline def oop: i20302.Opaque = 123

  object Main:
    def main(args: Array[String]): Unit =
      val o: Opaque = 123 // OK: does not compile
      val o2: Opaque = op // BUG: Does not compile because inferred type is int, but should compile
      val o3: Opaque = oop // OK: compiles

from #20302

[BalmungSan]

@som-snytt that doesn't seem to work here: https://scastie.scala-lang.org/BalmungSan/DzlQjmPBTuSZ08PIXqtbmw/54 unless I did something wrong.

[som-snytt]

@BalmungSan it checks for opaque proxies first. I don't know yet the difference in this simple example, however; I posted it here as a surprise.

[jchyb]

Interestingly, fully qualified names cause an opposite bug:

package i20302:

  opaque type Opaque = Int
  transparent inline def op: Opaque = 123
  transparent inline def oop: i20302.Opaque = 123

  object Main:
    def main(args: Array[String]): Unit =
      val o2: 123 = op // OK
      val o3: 123 = oop // BUG: Does not compile

Before inlining, the typer wraps the rhs of oop with a Typed tree (Typed(..., TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class i20302)),object XRepl$package),type Opaque)])), causing the "fix"/issue