Add Rust native symbolization library and C API wrapper

[christos68k]

This PR adds the Rust native symbolization framework that we developed at Elastic and now making available in OpenTelemetry as part of the ebpf-profiler project. We can use it to implement automatic on-target symbolization of native binaries (e.g. Go executables) but also to enable an OTel bulk native symbol upload protocol that we (Profiling SIG) can come together and specify.

Building through Cargo works and all tests are passing.

NOTE: This is the initial PR that introduces the Rust symbolization framework but does not resolve packaging and artifact questions. We may decide to split this out into a separate repository post-merge.

[korniltsev]

nit: Can we add target/ to .gitignore ?

[athre0z]

LGTM! Thanks so much for going through the effort of open-sourcing it!

NOTE: This is the initial PR that introduces the Rust symbolization framework but does not resolve packaging and artifact questions. We may decide to split this out into a separate repository post-merge.

Yeah: it would be best to have this in a separate repo at some point, but I also understand that this simplifies the contribution / donation process, so temporarily having it here is fine with me.

[korniltsev]

<3 Thank you for open sourcing this!

[korniltsev]

Suggested change

	eprintln!("Processing {:?}", &unit);
	debug!("Processing {:?}", &unit);

[christos68k]

There are a few such statements, leaving as-is for now and can switch to debug! once we start using the library from the agent.

[korniltsev]

Suggested change

	impl From<SymblibString> for Option<String> {
	fn from(maybe_str: SymblibString) -> Self {
	if maybe_str.0.is_null() {
	None
	} else {
	let cstr = unsafe { CString::from_raw(maybe_str.0) };
	mem::forget(maybe_str);
	Some(cstr.into_string().unwrap())
	}
	}
	}

I think it is not used.

[christos68k]

Leaving it in for now, can revisit in the future.

[korniltsev]

I know this first iteration is not immediately intended to be used in the profiler itself, however since there are ideas/plans to do this in the future I will just dump my thougths.
This API is not very convenient to use for ontarget symbolization because paths will likely be relative to /proc/{pid}/root. Maybe we can add an extra argument rootfs so that resolving debugaltlink is resolved in the same root fs. Or maybe as an alternative we can create an alternative API accepting file descriptors, so that rust lib does not need to worry about rootfs at all.

[christos68k]

Thanks, feel free to open an issue so that this doesn't get lost in the noise.

[korniltsev]

please correct me if I'm wrong, but symbfile format seems not suitable for quick address => symbol lookups, only for storing/transfer. Are there any plans to opensource the lookup data structure for use?

[christos68k]

symbfile is meant to be used for storage/transfer of symbol data. Efficient point lookups akin to blazesym is something we need to add to symblib. Essentially, we need both point lookups and bulk symbol extraction. Right now symblib does the latter but not the former and we're discussing how to proceed on getting that implemented.

[korniltsev]

I'm not sure if this file was hand written, but it may worth generating the header using cbindgen to be always in sync with rust code

[korniltsev]

Have you considered implementing small subset of the disassembler in a rust?
Asking because of security reasons. I would be cautious to run a disassembler written in C on potentially untrusted user data, especially if running as root during ontarget symbolization.

It would be nice at least to document somehow, that the amd64 retpad extractor is using C library for security visibility.

[athre0z]

Other than with ARM64, it's unfortunately not possible to implement a subset of the disassembler for AMD64. It's a variable length instruction set, so you need to be able to correctly decode each instruction in the program in order to find the start of the next one.