Stack overflow in compiler involving closures

[frewsxcv]

@jimtla pasted his code he was working with on IRC and I simplified it a bit:

#[derive(Clone)]
struct A (B);

impl A {
    pub fn matches<F: Fn()>(&self, f: &F) {
        let &A(ref term) = self;
        term.matches(f);
    }
}

#[derive(Clone)]
enum B {
    Variant1,
    Variant2(C),
}

impl B {
    pub fn matches<F: Fn()>(&self, f: &F) {
        match self {
            &B::Variant2(ref factor) => {
                factor.matches(&|| ())
            }
            _ => unreachable!("")
        }
    }
}

#[derive(Clone)]
struct C (D);

impl C {
    pub fn matches<F: Fn()>(&self, f: &F) {
        let &C(ref base) = self;
        base.matches(&|| {
            C(base.clone()).matches(f)
        })
    }
}

#[derive(Clone)]
struct D (Box<A>);

impl D {
    pub fn matches<F: Fn()>(&self, f: &F) {
        let &D(ref a) = self;
        a.matches(f)
    }
}

pub fn matches() {
    A(B::Variant1).matches(&(|| ()))
}

fn main() {}

/t/m/h/src (master|…) $ rustc lib.rs
lib.rs:18:36: 18:37 warning: unused variable: `f`, #[warn(unused_variables)] on by default
lib.rs:18     pub fn matches<F: Fn()>(&self, f: &F) {
                                             ^

thread 'rustc' has overflowed its stack
fish: Job 1, 'rustc lib.rs ' terminated by signal SIGILL (Illegal instruction)

This might be a duplicate of #21410

[ebfull]

I'm getting a similar stack overflow with closures. (I think this shouldn't be able to monomorphize?)

extern crate threadpool;

use std::sync::{Arc,Mutex};
use threadpool::ScopedPool;

fn test<'pool, F: FnMut(usize) + Send + Sync + 'pool>(mut callback: F, depth: usize, pool: Arc<Mutex<ScopedPool<'pool>>>) {
    if depth == 0 {
        callback(100);
    } else {
        struct Foo<F: FnMut(usize)> {
            callback: F
        }

        let foo = Arc::new(Mutex::new(Foo{callback: move |u| {
            callback(u);
        }}));

        for _ in 0..50 {
            struct Bar<F: FnMut(usize)> {
                callback: F
            }

            let foo = foo.clone();

            let bar = Arc::new(Mutex::new(Bar {
                callback: move |u| {
                    let foo = &mut *foo.lock().unwrap();

                    (foo.callback)(u);
                }
            }));

            for _ in 0..5 {
                let descending_pool = pool.clone();
                let pool = pool.lock().unwrap();
                let bar = bar.clone();
                pool.execute(move || {
                    test(move |n| {
                        let bar = &mut *bar.lock().unwrap();

                        (bar.callback)(n);
                    }, depth - 1, descending_pool);
                });
            }
        }
    }
}

fn main() {
    let pool = Arc::new(Mutex::new(ScopedPool::new(4)));

    test(|num| {
        println!("woohoo {}", num);
    }, 1, pool);
}

thread 'rustc' has overflowed its stack

[VladimirZhilyakov]

This code doesn't compile too.

struct Button<T: FnMut()> {
    f: Option<T>,
    title: &'static str,
}

impl<T: FnMut()> Button<T> {
    fn new() -> Button<T>
    { Button { f: None, title: "ClickMe" } }

    fn set(&mut self, f: T)
    { self.f = Some(f); }

    fn title(&mut self, title: &'static str)
    { self.title = title; }

    fn click(&mut self) {
        match self.f {
            Some(ref mut f) => f(),
            None => ()
        }
    }
}

fn main() {
    let mut btn = Button::new();
    btn.set(|| btn.title("Done"));
    btn.click();
    btn.click();
}

[frewsxcv]

Status update: my code no longer has an ICE, but instead has:

<anon>:5:5: 8:6 error: reached the recursion limit during monomorphization
<anon>:5     pub fn matches<F: Fn()>(&self, f: &F) {
<anon>:6         let &A(ref term) = self;
<anon>:7         term.matches(f);
<anon>:8     }
playpen: application terminated with error code 101

[ebfull]

@frewsxcv That's good. Most of these code snippets should result in that sort of error. The type inference is hiding a recursive type definition which will necessarily arise during monomorphization. These are impossible to express statically.

By the way, here's a reduced example of the stack overflow:

fn main() {
    let mut test = None;
    test = Some(|| test.unwrap());
}

I think because it's otherwise impossible to express recursive type definitions in code, a lot of different areas of the compiler assume this will not happen. But with closures it can right now; we should probably check for that somewhere.

[pmarcelll]

This seems to be fixed in nightly.