Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Announcing the FFI-unwinding Project Group #2797

Merged
merged 10 commits into from
Dec 12, 2019
Merged

Announcing the FFI-unwinding Project Group #2797

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7400ec9
Copy project-group RFC text from 'project-ffi-unwind' repo
Oct 28, 2019
abcfe9d
Apply suggestion from Centril
BatmanAoD Oct 28, 2019
8c8da9a
Fix tense
BatmanAoD Oct 28, 2019
b55aa0f
Centril - mention that lang team will review RFCs/FCPs
BatmanAoD Oct 28, 2019
96b6735
Centril & gnzlbg: No constraints on any target
BatmanAoD Oct 29, 2019
9de62c2
Coercions != subtyping
BatmanAoD Oct 29, 2019
cd6c600
Copy language-feature goals from charter doc
BatmanAoD Oct 29, 2019
81dd0bb
clarify what we intend to specify for foreign exceptions
nikomatsakis Dec 12, 2019
73fa873
rewrite the "TBD behavior" section
nikomatsakis Dec 12, 2019
1fe0929
rename to RFC 2797
nikomatsakis Dec 12, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
211 changes: 211 additions & 0 deletions text/0000-project-ffi-unwind.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
- Feature Name: `project-unwind-FFI`
- Start Date: 2019-10-27
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)

# Summary
[summary]: #summary

* To create a "project group" with the purpose of designing subsequent RFCs to
flesh out the details of the "C unwind" ABI
BatmanAoD marked this conversation as resolved.
Show resolved Hide resolved
* The "project group" term is newly introduced: it is a specific type of
working group whose goal is to flesh out a particular proposal or complete
a project.
* This project group plans to specify how "C unwind" works on major
BatmanAoD marked this conversation as resolved.
Show resolved Hide resolved
platforms.
* The primary goal is to enable Rust panics to propagate safely across
foreign frames.
* A future goal may be to enable foreign exceptions to propagate across Rust
frames.
* We do not plan to allow catching or throwing foreign exceptions from Rust
code

# Motivation
[motivation]: #motivation

Unwinding through Rust's `extern "C"` ABI is [Undefined Behavior]. There is an
[existing plan][abort-unwind] to make the behavior of Rust's `panic`
well-defined by causing Rust functions defined with `extern "C"` to abort the
application whenever an uncaught `panic` would otherwise escape into the
caller. Unfortunately, previous attempts to stabilize this behavior have caused
existing, working projects to break.

The problem here is not that the existing projects break *per se*: they are
relying on [Undefined Behavior], so breakage is to be expected as a
possibility. The problem is that there is no alternative available to them that
would allow them to keep working (even if they are continuing to rely on
behavior that is not yet fully specified).

Previous attempts to provide a well-defined mechanism for unwinding across FFI
boundaries have failed to reach consensus. Notably, two proposed RFCs generated
over 400 comments between them before ultimately being closed:

* [rust-lang/rfcs#2699](https://github.com/rust-lang/rfcs/pull/2699)
* [rust-lang/rfcs#2753](https://github.com/rust-lang/rfcs/pull/2753)

GitHub comment threads become difficult to follow for discussions this lengthy,
and the disagreements in these threads have felt less productive than we
believe they could be if more structure were provided.

We would also like to demonstrate the Rust lang team's commitment to providing
such a mechanism without needing to agree in advance on what language changes
will be stabilized in order to do so.
Centril marked this conversation as resolved.
Show resolved Hide resolved

# Prototyping 'shepherded' project groups
[prototyping-project-groups]: #prototyping-shepherded-project-groups

With this RFC, we formally announce the formation of a project-specific,
shepherded "project group" to adopt responsibility for driving progress on
specifying unwinding behavior at FFI boundaries.

## What is a "project group"?

The "project group" term has not previously been used: it is intended to
formalize a concept that has existed informally for some time, under a number
of names (including "working group").

A "project group" is a group of people working on a particular project at the
behest of an official Rust team. Project groups must have:

* A **charter** defining the project's scope
* A **liaison** with an official Rust team (who may or may not also be a shepherd)
* A small number of **shepherds**, who are responsible for summarizing
conversations and keeping the lang team abreast of interesting developments.
* A GitHub repository hosted under the `rust-lang` organization containing the
charter and instructions for how community members can monitor the group's
progress and/or participate.

[This blog post][shepherds-3.0] explains in detail the role of the
shepherds.

## Project group roadmap and RFCs

The first step of the project group is to define a **roadmap** indicating the
planned sequence in which it will design and propose particular behaviors and
features. Once the project group feels it has completed work on some item in
the roadmap, that item will be submitted to the community as an RFC or FCP.
BatmanAoD marked this conversation as resolved.
Show resolved Hide resolved

## Stabilizing unspecified "TBD" behavior
[stabilizing-tbd]: stabilizing-unspecified-tbd-behavior

We would like to be able to provide features in stable Rust without a full
formal specification but with an informal statement of intent regarding the
behavior. These features would be considered "unspecified behavior" (rather
than "undefined behavior"), and their behavior would therefore be subject to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As worded right now, this sort of assumes that everyone agrees what "unspecified" means (that often turns out not to be the case). I assume this will be detailed and clarified as the project progresses. I hope this will not turn into "implementation defined" or some such.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have indeed written up our own definition, and I think it would be worthwhile to open an RFC to concretely define this terminology.

I think "implementation defined" would be a good addition to the spec-terminology doc.

Copy link
Contributor

@gnzlbg gnzlbg Oct 29, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative is for "TBD" to be an orthogonal denotation to whatever the behavior class used by the reference is (unspecified, undefined, target-dependent, ...).

One of the consequences of unspecified meaning different things to different people (and different WGs, the UCGs has its own definitions and they are controversial!), is that users might end up confused on whether "unspecified" behavior can become "undefined" in the future or not.

I don't think that's something that's worth resolving right now, and I think that it is in the best interest of the Project Group to have as much freedom as possible. By making it an orthogonal annotation, the Project Group can say that "X is UB (TBD) and Y is target-dependent (TBD)" meaning that X is UB and Y is target dependent, and that the project group is trying to improve both, but while X might remain UB, Y won't become worse than target-dependent (e.g. Y won't become UB, but it might become a well-defined thing on all platforms).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I appreciate the suggestion, @gnzlbg, but I don't really think that helps with the problem Niko and I were trying to solve by defining/adopting "TBD", which is to have a way of saying, essentially, "this is not UB in the sense that the compiler may optimize based on the assumption that it can't happen; we have not fully specified the behavior, but you can expect it to 'work' approximately as you'd expect, and we promise to further specify the behavior later (though until that time the details of the behavior-as-implemented may change)."

I don't think it would be appropriate to define "unspecified behavior" as part of this "announcement" RFC, but if @Centril (or someone else) would prefer, I can just remove this section entirely (assuming @nikomatsakis agrees that's okay).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would mean that any behavior that we might want to define in some way, and one of the alternatives being considered is defining it as "UB", would need to be UB instead of TBD. If that's the intent, then that's fine with me. Maybe add a sentence mentioning just that ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BatmanAoD I'm just happy with the knowledge that you'll take into account that I don't think this should become implementation-defined and that the lang team should have the freedom to change the Rust unwind ABI on any platform including with shims if necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnzlbg Yes, I think that's the intent; I would say that nothing can be "TBD" unless it's got an "approximate" or "informal" behavior spec that we don't intend to change. The canonical example is "extern "C unwind" will do 'the right thing' on any platform." That's not well-defined behavior; it's not even implementation defined. But it does express a (hopefully shared) understanding of what code examples should be expected to work, and a statement of intent from the lang team that such code won't break if and when the behavior is further specified. @nikomatsakis would you agree with this explanation?

If so, I can add some verbiage to this effect.

Copy link
Contributor

@nikomatsakis nikomatsakis Oct 30, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My two cents:

I think it is useful to enumerate our scope and goals and to communicate those goals to people. So I think that distinguishing TBD is very useful, as it communicates that. I think that if we can give people guidelines as to our intent (along the lines that @BatmanAoD sketched -- perhaps more specific) that's even better.

That said, I think we should leave ourselves the freedom to make decisions. I think we can define our intent here as roughly:

  • We intend to make it possible for Rust panics and foreign exceptions to propagate through a "C unwind" boundary in a well-defined and specified way.
  • As a non-exhaustive list, this means that we intend to eventually use cases like the following:
    • a C++ exception unwinding Rust frames, running destructors as it goes
    • a Rust panic unwinding through C++ frames, running destructors as it goes
    • a Rust panic unwinding into C++ and then back into Rust, and being caught via catch_unwind
  • However, there are a number of details whose resolution is not clear. As a non-exhaustive list:
    • what should happen when a C++ exception attempts to unwind through a Rust catch_unwind call?
    • how should longjmp, which is defined on some platforms via unwinding, interact with frames that contain destructors? (And is that even in scope for this group?)

Along those lines, I think we can specify that:

  • For Tier 1 platforms, there will be some translation from Rust panics into the native unwinding mechanism, but we have not yet specified how Rust panics will be described
  • Similarly, we have not specified what the behavior will be when unwinding across a "C unwind" barrier occurs, except for the case where the exception was originally a Rust panic, in which case it is translated back into a Rust panic
    • This implies that we have not specified what happens when catch_unwind is involved, since before catch_unwind gets involved, there must be a "C unwind" barrier.
  • We expect this specification work to be done on a per-platform basis

I'm going to be honest and say that the distinction between "unspecified behavior" and "undefined behavior" feels like a "distinction without a difference" to me, most of the time, unless that unspecified behavior is further qualified to some narrower range of possibilities. I guess I would mostly rather we just don't waste a lot of time (too late...) arguing back and forth on this, as I'm not sure what practical import it has. I'd rather we just spend our time deciding what the behavior ought to be.

some change, but such a change would be guaranteed not to violate the project
group's stated intent for the feature.

Internally, a project group should refer to such unspecified behavior as "TBD",
to indicate that refining the specification of the behavior is within the scope
of the project. Outside of the context of the project group, such as in general
reference material for the language, the behavioral intent as determined by the
project group should be described, but will not imply any special status
compared to other instances of unspecified behavior in the language.

## Details of the FFI-unwind project group

[Repository][ffi-unwind project]

Initial shepherds:

* [acfoltzer (Adam)](https://github.com/acfoltzer)
* [batmanaod (Kyle)](https://github.com/batmanaod)

Lang team liaisons:

* [nikmoatsakis (Niko)](https://github.com/nikmoatsakis)
* [joshtriplett (Josh)](https://github.com/joshtriplett)

### Charter
[charter]: #charter

The FFI-unwind project group has the following initial scope:

* to define the details of the "C unwind" ABI on major Tier 1 platforms
* in particular, to define with sufficient detail to enable the use cases
described in the Motivation section of this RFC

Certain elements are definitively out of scope:

* The group does not intend to consider mechanisms to enable "interop"
between Rust panics and exceptions from other languges. For example,
we do not intend to permit Rust code to catch C++ exceptions, though
we will have to consider what happens when a C++ exception unwinds
past a `catch_unwind` boundary.
nikomatsakis marked this conversation as resolved.
Show resolved Hide resolved

### Constraints and considerations

In its work, the project-group should consider various constraints and
considerations:

* The possibility that C++ may adopt new unwinding mechanisms in the future.
* The possibility that Rust may alter its unwinding mechanism in the future --
in particular, the project group must not propose a design that would
constrain Rust's unwinding implementation.
BatmanAoD marked this conversation as resolved.
Show resolved Hide resolved

### Participation in the project group

Like any Rust group, the FFI-unwind project group intends to operate
in a public and open fashion and welcomes participation. Visit the
[repository][ffi-unwind project] for more details.

# Drawbacks
[drawbacks]: #drawbacks

* The adoption of project groups for major language design efforts is a change
in the status quo. We believe that this change will be an improvement over
the current RFC-centric process, but we should be wary of unintended
consequences of from such a change.
* [Stabilization of "TBD" features][stabilizing-tbd] may be surprising or
confusing to users, and it will encourage reliance on (some) unspecified
behavior.

# Prior art
[prior-art]: #prior-art

Although the term "project group" is new, some existing efforts, such as the
Unsafe Code Guidelines effort and the work around defining const evaluation,
were organized in a similar fashion.

In addition to the [blog post Niko Matsakis][shepherds-3.0] about
shepherding, James Munns wrote a [more formal shepherding
proposal][shepherding-3.1].

The [governance WG][governance-wg] and [lang-team meta working
group][lang-meta-wg] were both formed at least in part to improve the process
for large-scale design efforts. One existing proposal is for ["staged
RFCs"][staged-rfc]; this may be considered a precursor to the current
"shepherded project group" proposal.


# Unresolved questions and Future possibilities
[unresolved-questions]: #unresolved-questions

Since this RFC merely formalizes the creation of the project group, it
intentionally leaves all technical details within the project's scope
unresolved.

# Future possibilities
[future-possibilities]: #future-possibilities

The project group will start with a fairly [limited scope][charter], but if the
initial effort to design and stabilize a safe cross-language unwinding feature
on a limited set of platforms goes well, there are many related areas of
potential exploration. Three noteworthy examples are:

* Catching foreign unwinding (e.g. Rust catching C++ exceptions, or C++
catching Rust `panic`s)
* Defining subtype relationships among `fn`s using ABIs with different `unwind`
BatmanAoD marked this conversation as resolved.
Show resolved Hide resolved
behavior
* Monitoring progress, or even participating in discussion about, the [ISO C and
C++ proposal][c-cpp-unified-proposal] for cross-language error handling

[Undefined Behavior]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html
[abort-unwind]: https://github.com/rust-lang/rust/issues/52652
[ffi-unwind project]: https://github.com/rust-lang/project-ffi-unwind
[shepherds-3.0]: http://smallcultfollowing.com/babysteps/blog/2019/09/11/aic-shepherds-3-0/
[c-cpp-unified-proposal]: http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1095r0.pdf
[shepherding-3.1]: https://jamesmunns.com/blog/shepherding-3-1/
[governance-wg]: https://github.com/rust-lang/wg-governance
[lang-meta-wg]: https://github.com/rust-lang/lang-team/tree/master/working-groups/meta
[staged-rfc]: http://smallcultfollowing.com/babysteps/blog/2018/06/20/proposal-for-a-staged-rfc-process/