Skip to content
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "TermInterface"
uuid = "8ea1fca8-c5ef-4a55-8b96-4e9afe9c9a3c"
authors = ["Shashi Gowda <gowda@mit.edu>", "Alessandro Cheli <sudo-woodo3@protonmail.com>"]
version = "0.3.3"
version = "0.4"

[compat]
julia = "1"
Expand Down
138 changes: 91 additions & 47 deletions src/TermInterface.jl
Original file line number Diff line number Diff line change
@@ -1,14 +1,36 @@
"""
This module defines a contains definitions for common functions that are useful
for symbolic expression manipulation. Its purpose is to provide a shared
interface between various symbolic programming Julia packages.

This is currently borrowed from TermInterface.jl. If you want to use
Metatheory.jl, please use this internal interface, as we are waiting that a
redesign proposal of the interface package will reach consensus. When this
happens, this module will be moved back into a separate package.

See https://github.com/JuliaSymbolics/TermInterface.jl/pull/22
"""
module TermInterface

"""
istree(x)

Returns `true` if `x` is a term. If true, `operation`, `arguments`
must also be defined for `x` appropriately.
Returns `true` if `x` is a term. If true, `head`, `children` and
`is_function_call` must also be defined for `x` appropriately.
"""
istree(x) = false
export istree

"""
is_function_call(x)

Returns true if a term abstractly represents a function call or function application.
Must be defined if `istree(x)` is defined.
Can be true only if `istree(x)` is true.
"""
function is_function_call end

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not default this to false? If it's an optional interface, why do we require users to define it?

@0x0f0f0f 0x0f0f0f Jan 14, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not make this optional. Defaulting it to false may cause users to forget to set it on specific term types, and thus cause hard to catch bugs as it would propagate through all the rewriting steps (it happened to me and i lost a few hours just to understand i was dispatching on the wrong type and was defaulting to false).

In comparison to the previous state of this redesign, where we had Head types, in this latest version, the provider of a term types T <: AbstractT just has to set is_function_call(<:AbstractT) = true if the whole language is functional, or set it to false if the entire AST does not have function calls

export is_function_call

"""
symtype(x)

Expand All @@ -22,6 +44,7 @@ function symtype(x)
end
export symtype


"""
issym(x)

Expand All @@ -31,46 +54,32 @@ on `x` and must return a Symbol.
issym(x) = false
export issym

"""
exprhead(x)

If `x` is a term as defined by `istree(x)`, `exprhead(x)` must return a symbol,
corresponding to the head of the `Expr` most similar to the term `x`.
If `x` represents a function call, for example, the `exprhead` is `:call`.
If `x` represents an indexing operation, such as `arr[i]`, then `exprhead` is `:ref`.
Note that `exprhead` is different from `operation` and both functions should
be defined correctly in order to let other packages provide code generation
and pattern matching features.
"""
function exprhead end
export exprhead

head(x)

If `x` is a term as defined by `istree(x)`, `head(x)` returns the head of the
term. If `x` represents a function call term like `f(a,b)`, the head
is the function being called, `f`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly what this redesign had hoped to avoid! We don't want head to be the function being called. We want it to be :call and we want children(x)[1] to be the function being called.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we wanted to avoid the concept of :call as "exprhead".

In my latest commit head is the old operation, and children is the old arguments.

If we add both head,children and operation,arguments Metatheory.jl would just rely on operation and arguments for pattern matching. I guess the same for SU.

But what about SymbolicUtils terms? t = f(a,b) in SU would have operation(t) == f, arguments(t) == [a,b], children(t) = [f,a,b], what about head(t)? Should it be SUHead()?

I kinda dislike the idea that the users should define a struct to define the head of an AST node, all the information required to inspect, manipulate and create new terms is already contained in the type of the term.

"""
operation(x)
function head end
export head

If `x` is a term as defined by `istree(x)`, `operation(x)` returns the
head of the term if `x` represents a function call, for example, the head
is the function being called.
"""
function operation end
export operation

"""
arguments(x)
children(x)

Get the arguments of `x`, must be defined if `istree(x)` is `true`.
Get the children of a term `x`, must be defined if `istree(x)` is `true`.
"""
function arguments end
export arguments

function children end
export children

"""
unsorted_arguments(x::T)
unsorted_children(x::T)

If x is a term satisfying `istree(x)` and your term type `T` orovides
and optimized implementation for storing the arguments, this function can
be used to retrieve the arguments when the order of arguments does not matter
If x is a term satisfying `istree(x)` and your term type `T` provides
and optimized implementation for storing the children, this function can
be used to retrieve the children when the order of arguments does not matter
but the speed of the operation does.
"""
unsorted_arguments(x) = arguments(x)
Expand All @@ -80,10 +89,10 @@ export unsorted_arguments
"""
arity(x)

Returns the number of arguments of `x`. Implicitly defined
if `arguments(x)` is defined.
Returns the number of children of `x`. Implicitly defined
if `children(x)` is defined.
"""
arity(x) = length(arguments(x))
arity(x)::Int = length(children(x))
export arity


Expand All @@ -102,27 +111,62 @@ export metadata
Returns a new term which has the structure of `x` but also has
the metadata `md` attached to it.
"""
function metadata(x, data)
error("Setting metadata on $x is not possible")
end
function metadata(x, data) end


"""
similarterm(x, head, args, symtype=nothing; metadata=nothing, exprhead=:call)
maketerm(T::Type, head, children; is_call = true, type=Any, metadata=nothing)

Has to be implemented by the provider of the expression type T.
Returns a term that is in the same closure of types as `T`,
with `head` as the head and `children` as the arguments, `type` as the symtype
and `metadata` as the metadata.

Returns a term that is in the same closure of types as `typeof(x)`,
with `head` as the head and `args` as the arguments, `type` as the symtype
and `metadata` as the metadata. By default this will execute `head(args...)`.
`x` parameter can also be a `Type`. The `exprhead` keyword argument is useful
when manipulating `Expr`s.
`is_call` is used to determine if the constructed term represents a function
call. If `is_call = true`, then it must construct a term `x` such that
`is_function_call(x) = true`, and vice-versa for `is_call = false`.
"""
function similarterm(x, head, args, symtype = nothing; metadata = nothing, exprhead = nothing)
head(args...)
end
function maketerm end
export maketerm



export similarterm
"""
node_count(t)
Count the nodes in a symbolic expression tree satisfying `istree` and `arguments`.
"""
node_count(t) = istree(t) ? reduce(+, node_count(x) for x in children(t), init in 0) + 1 : 1
export node_count

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, though I wonder if AbstractTrees could help here.

"""
@matchable struct Foo fields... end [HeadType]

include("utils.jl")
Take a struct definition and automatically define `TermInterface` methods.
`is_function_call` of such type will default to `true`.
"""
macro matchable(expr)
@assert expr.head == :struct
name = expr.args[2]
if name isa Expr
name.head === :(<:) && (name = name.args[1])
name isa Expr && name.head === :curly && (name = name.args[1])
end
fields = filter(x -> x isa Symbol || (x isa Expr && x.head == :(::)), expr.args[3].args)
get_name(s::Symbol) = s
get_name(e::Expr) = (@assert(e.head == :(::)); e.args[1])
fields = map(get_name, fields)

quote
$expr
TermInterface.istree(::$name) = true
TermInterface.is_function_call(::$name) = true
TermInterface.head(::$name) = $name
TermInterface.children(x::$name) = getfield.((x,), ($(QuoteNode.(fields)...),))
TermInterface.arity(x::$name) = $(length(fields))
Base.length(x::$name) = $(length(fields) + 1)
end |> esc
end
export @matchable

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach to building ASTs results in huge slowdowns in compilers, because compilers specialize on each combination of node types. I appreciate the idea of adding an optional AST easy implementation feature, could we add it in a separate PR?

A few ideas:

  • enforce symbols or enums to differentiate heads in the AST (runtime not inference time)
  • make it clear that head_symbol is not part of the TermInterface interface, it's just part of the easy-mode ast builder.
  • add a simplified approach towards types with dynamic field information. Because we would emit types that have the same type for every head, we would need to use dynamic field information.

If we feel like an optimized AST implementation is out of scope, we shouldn't include programming patterns that will inconvenience users later on with performance issues.

include("expr.jl")

Expand Down
33 changes: 18 additions & 15 deletions src/expr.jl

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file seems like the right approach to me

Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@

# This file contains default definitions for TermInterface methods on Julia
# Builtin Expr type.

istree(x::Expr) = true
exprhead(e::Expr) = e.head
is_function_call(e::Expr) = _is_function_call_expr_head(e.head)
_is_function_call_expr_head(x::Symbol) = x in (:call, :macrocall)

operation(e::Expr) = expr_operation(e, Val{exprhead(e)}())
arguments(e::Expr) = expr_arguments(e, Val{exprhead(e)}())
istree(x::Expr) = true

# See https://docs.julialang.org/en/v1/devdocs/ast/
expr_operation(e::Expr, ::Union{Val{:call},Val{:macrocall}}) = e.args[1]
expr_operation(e::Expr, ::Union{Val{:ref}}) = getindex
expr_operation(e::Expr, ::Val{T}) where {T} = T

expr_arguments(e::Expr, ::Union{Val{:call},Val{:macrocall}}) = e.args[2:end]
expr_arguments(e::Expr, _) = e.args
head(e::Expr) = is_function_call(e) ? e.args[1] : e.head
children(e::Expr) = is_function_call(e) ? e.args[2:end] : e.args

function arity(e::Expr)::Int
l = length(e.args)
is_function_call(e) ? l - 1 : l
end

function similarterm(x::Expr, head, args, symtype = nothing; metadata = nothing, exprhead = exprhead(x))
expr_similarterm(head, args, Val{exprhead}())
function maketerm(T::Type{Expr}, head, children; is_call=true, type=Any, metadata=nothing)
if is_call
Expr(:call, head, children...)
else
Expr(head, children...)
end
end

maketerm(T::Type{Expr}, head::Union{Function,DataType}, children; is_call=true, type=Any, metadata=nothing) =
maketerm(T, nameof(head), children; is_call, type, metadata)

expr_similarterm(head, args, ::Val{:call}) = Expr(:call, head, args...)
expr_similarterm(head, args, ::Val{:macrocall}) = Expr(:macrocall, head, args...) # discard linenumbernodes?
expr_similarterm(head, args, ::Val{eh}) where {eh} = Expr(eh, args...)
16 changes: 0 additions & 16 deletions src/utils.jl

This file was deleted.

67 changes: 56 additions & 11 deletions test/runtests.jl
Original file line number Diff line number Diff line change
@@ -1,17 +1,62 @@
using TermInterface
using Test
using TermInterface, Test

@testset "Expr" begin
ex = :(f(a, b))
@test operation(ex) == :f
@test arguments(ex) == [:a, :b]
@test exprhead(ex) == :call
@test ex == similarterm(ex, :f, [:a, :b])
@test istree(ex)
@test is_function_call(ex)
@test head(ex) == :f
@test children(ex) == [:a, :b]
@test ex == maketerm(Expr, :f, [:a, :b])

ex = :(arr[i, j])
@test operation(ex) == getindex
@test arguments(ex) == [:arr, :i, :j]
@test exprhead(ex) == :ref
@test ex == similarterm(ex, :ref, [:arr, :i, :j]; exprhead = :ref)
@test ex == similarterm(ex, :ref, [:arr, :i, :j])
@test istree(ex)
@test !is_function_call(ex)
@test head(ex) == :ref
@test children(ex) == [:arr, :i, :j]
@test ex == maketerm(Expr, :ref, [:arr, :i, :j]; is_call=false)


ex = :(i, j)
@test istree(ex)
@test !is_function_call(ex)
@test head(ex) == :tuple
@test children(ex) == [:i, :j]
@test ex == maketerm(Expr, :tuple, [:i, :j]; is_call=false)

ex = Expr(:block, :a, :b, :c)
@test istree(ex)
@test !is_function_call(ex)
@test head(ex) == :block
@test children(ex) == [:a, :b, :c]
@test ex == maketerm(Expr, :block, [:a, :b, :c]; is_call=false)
end

@testset "Custom Struct" begin
struct Foo
args
Foo(args...) = new(args)
end
TermInterface.istree(::Foo) = true
TermInterface.is_function_call(::Foo) = true
TermInterface.head(::Foo) = Foo
TermInterface.children(x::Foo) = collect(x.args)

t = Foo(1, 2)
@test istree(t)
@test is_function_call(t)
@test head(t) == Foo
@test children(t) == [1, 2]
end

@testset "Automatically Generated Methods" begin
@matchable struct Bar
a
b::Int
end

t = Bar(1, 2)
@test istree(t)
@test is_function_call(t)
@test head(t) == Bar
@test children(t) == (1, 2)
end