Support custom target-spec

[boozook]

Could be great to support custom target-specs (json files).

As we all know, env TARGET contains rustc's "short target name", not exactly target-triple. It can be triple only if builtin target requested.
So cc couldn't use it directly if some custom target requested, e.g. for --target=my-spec.json env TARGET will be "my-spec".

How to do it:

1. change there target type - wrap into something like enum Target { Builtin(...), Custom(...) }
2. probe target to determine is it builtin or not and get details:

• we have env RUSTC by cargo
• $RUSTC --print cfg --target {target-short-name}.json
• $RUSTC --print target-spec-json --target {target-short-name}.json -Zunstable-options (nightly only)
  From this step we can get llvm-target

But actually we already have almost all we need by cargo's env:

• CARGO_CFG_TARGET_ABI
• CARGO_CFG_TARGET_ARCH
• CARGO_CFG_TARGET_FEATURE
• CARGO_CFG_RELOCATION_MODEL
• other vars starting with CARGO_CFG_TARGET_ excluding _VENDOR and _OS, and probably _ENV.

Note, CARGO_CFG_TARGET_FEATURE contains features added in the my-spec.json too, so it really useful.

So, could be great to support target-json-specs, use info from them.
Also to respect and use all available data from vars CARGO_CFG_TARGET_ - translate and pass to compiler if possible.

Example of env vars given by cargo

In my case, my target.json contains "llvm-target": "thumbv7em-none-eabihf", uses it as base target, but overrides features and some more.

TARGET: "my-spec"
CARGO_CFG_OVERFLOW_CHECKS: ""
CARGO_CFG_PANIC: "abort"
CARGO_CFG_RELOCATION_MODEL: "pic"
CARGO_CFG_TARGET_ABI: "eabihf"
CARGO_CFG_TARGET_ARCH: "arm"
CARGO_CFG_TARGET_ENDIAN: "little"
CARGO_CFG_TARGET_ENV: "elf"
CARGO_CFG_TARGET_FEATURE: "dsp,mclass,thumb-mode,thumb2,v5te,v6,v6k,v6t2,v7,neon"
CARGO_CFG_TARGET_HAS_ATOMIC: "16,32,8,ptr"
CARGO_CFG_TARGET_HAS_ATOMIC_EQUAL_ALIGNMENT: "16,32,8,ptr"
CARGO_CFG_TARGET_HAS_ATOMIC_LOAD_STORE: "16,32,8,ptr"
CARGO_CFG_TARGET_OS: "myneatos"
CARGO_CFG_TARGET_POINTER_WIDTH: "32"
CARGO_CFG_TARGET_THREAD_LOCAL: ""
CARGO_CFG_TARGET_VENDOR: "custom"
CARGO_CFG_UB_CHECKS: ""
CARGO_ENCODED_RUSTFLAGS: ""
RUSTC: "path/to/rustc"

Probably related issue: #994

[NobodyXu]

Also to respect and use all available data from vars CARGO_CFG_TARGET_ - translate and pass to compiler if possible.

Yeah that sounds reasonable.

Supporting target spec json query though is a bit more complicated, I suppose we can add the API to pass that information to cc, and do the parsing in another crate, since it would require JSON parsing and maybe nightly compiler.

[boozook]

That could be really helpful!

[boozook]

Also things like that is not work for target with custom names, definitely. Imagine I'll name my target arch-foo-bar-like-msvc or more funny same-but-not-msvc :))) , or better for realistic example msvc-plugin-for-arch-gnu :).

That things could be good practice to determine by compiler (print), or by cargo (env), or directly from json.

Anyway about that concert msvc - I suppose will be better to get it from CARGO_CFG_TARGET_ENV - if value is known, so ok; otherwise just ignoring unknown value.

Wow, just search in the lib.rs for target.contains. I see 125 findings.

---

Also cc fails on target-names that doesn't contains dash, e.g. "strangecpu", trying to split it by -.

[NobodyXu]

Wow, just search in the lib.rs for target.contains. I see 125 findings.

Yeah it is a bit messy

[boozook]

I did some research and experiments and here are my thoughts on the topic.

Build::target:

• sorry, currently totally incompatible with name of user's custom target, that is just file-stem (as mentioned above it is rustc's "short target name", not a triple), so we have to interpret it only as random identifier - do not try parse, split and get any useful info from it;
• should be absolutely optional in case of it is user's custom target (reed explanation below)

In the case of TARGET is custom, I propose instead of Build::target to add and use methods for abi, arch, endian and others. We can automatically set it by env vars CARGO_CFG_TARGET_ received by cargo. And so user will be able to set/override it calling those methods (same as currently target).

We have to use all env vars CARGO_CFG_TARGET_, that necessary for our target configuration, forgetting about target-triple. Next is for clang-like for example:

• set -march={} from CARGO_CFG_TARGET_ARCH
• set -mbig-endian if CARGO_CFG_TARGET_ENDIAN is "big"
• -mabi={} from CARGO_CFG_TARGET_ABI
  • if ends with hf like eabihf split and set -mabi= and -mhard-float
• -mcpu={} from nowhere 🤷🏻‍♂️ (see Open Questions)

We have to do translation of values from env vars to names specially for specified (or auto-determined) compiler, you know.

Same for CARGO_CFG_TARGET_POINTER_WIDTH and CARGO_CFG_RELOCATION_MODEL.

Important: if target is already known and set, then vars like CARGO_CFG_TARGET_ARCH must override values we set to compiler. It because llvm-target is the base, but necessary values in the spec (arch, abi, etc..).

Hard part

Values from CARGO_CFG_TARGET_FEATURE, CARGO_CFG_TARGET_HAS_ATOMIC, CARGO_CFG_TARGET_HAS_ATOMIC_LOAD_STORE, CARGO_CFG_TARGET_HAS_ATOMIC_EQUAL_ALIGNMENT should be parsed and translated for chosen compiler.

Some notes

We definitely shouldn't try to search for a file in the file system and read it. It's not reliable, fragile. There is no guarantee that we will find it (for example, it can initially be passed to cargo as follows: --target=../file.json (relative path points to outside), but at run-time of build-script PWD can (actually will) be other then when user called cargo, especially in the case of workspace.

Open questions

1. How to accurately 100% reliably determine that we have a user target?
2. How to determine CPU automatically

I'm working on proposal and patch for cargo to add new envs that exposes some necessary data for us - "is the requested target is user's spec file json?", "specified CPU" (can be or not), LLVM-target-triple (from json) and maybe path to json-spec (The latter is very dubious).

@NobodyXu,
Do you have some suggestions?

[NobodyXu]

I think it's pretty good, I recommend to split it into multiple PRs and do it incrementally.

[boozook]

Firstly I filled proposal-issue rust-lang/cargo/issues 14208.
You could thumb-up it and comment with your suggestions, could be great 😊

[boozook]

I think I understood something. Perhaps we have enough information for a complete configuration of a third-party compiler.

We don't need CPU because we have to use minimal-basic-cpu as the base. Kinda generic.

Same for llvm-target. We already had abi and arch.

We already know all target-features, that only need to be translated for a compiler (clang, gcc, etc..)

Also we have data-layout - sizes of pointer and atomics. What about aligning and padding that also have to be known? 🤷🏻‍♂️

But any info about linking with std and buildins is not available. What we should to do with it? 🤷🏻‍♂️

Just need to try.

[boozook]

Well, I'll try to implement some part of it, but I need to make target field optional.

[madsmtm]

#1225 does most of the work here, the remaining work is to eliminate usage of get_raw_target