Add static initializer support to bpf2c - part 1

[Alan-Jowett]

Description

Add support for static map initialization via BTF map definitions.

This is part 1, which only changes bpf2c to emit the map definitions in the compiled PE image and to the native loader to consume it.

Part 2 will update the ELF loader to consume this data and set the initial values in the maps.

Testing

CI/CD

Documentation

No.

Installation

No.

Map's data is stored in the ".maps" section of the ELF file. The symbols for the ".maps" section gives the starting and ending offset of each map. The relocations for the ".maps" section give the offset of the initial values for each map. Each relocation record is a pair of (offset, symbol) where the symbol is the map value to insert. To convert offset to index in the "values" field, the first step is to determine which map the offset is for. This is done by finding the map whose range contains the offset. Then the offset is converted to an index by subtracting the offset of the values array and dividing by the size of a pointer. Finally, the value is inserted into the map's initial values vector at the computed index.

As an example, assume a map declared as:

{
    __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
    __uint(max_entries, 3);
    __uint(key_size, sizeof(uint32_t));
    __array(values, int(struct xdp_md* ctx));
} map SEC(".maps") = {
    // First and last entries are NULL to test that we can handle cases where
    // the initial values are not at the beginning of the array.
    .values =
        {
            NULL,
            recurse,
            NULL,
        },
};

This produces the following symbol records:

D:\ebpf-for-windows\x64\Debug>llvm-objdump --syms tail_call_recursive.o --section=.maps | findstr .maps
llvm-objdump.exe: warning: section '.maps' mentioned in a -j/--section option, but not found in any input file
0000000000000030 g     O .maps  0000000000000020 canary
0000000000000000 g     O .maps  0000000000000030 map

So, the maps are as follows:
(0x0, 0x30) -> map
(0x30, 0x50) -> canary.

The relocation data for the .maps section then contains:

tail_call_recursive.o:  file format elf64-bpf

RELOCATION RECORDS FOR [.maps]:
OFFSET           TYPE                     VALUE
0000000000000020 R_BPF_64_64              recurse

So, the relocation is at offset 0x20, which is within the map called "map".
The offset of the "values" field is 0x18 (from the BTF data, not shown here).

Index is then:
(0x20 - 0x18) / sizeof(uintptr) -> 1

So, the map value at index 1 should be "recurse".

There are no relocation records for NULL entries.

Partially resolves: #2555

[codecov]

Codecov Report

Merging #2728 (9fe2555) into main (535bfe2) will increase coverage by 0.02%.
Report is 11 commits behind head on main.
The diff coverage is 86.78%.

❗ Current head 9fe2555 differs from pull request most recent head 86ca22d. Consider uploading reports for the commit 86ca22d to get more accurate results

@@            Coverage Diff             @@
##             main    #2728      +/-   ##
==========================================
+ Coverage   87.19%   87.21%   +0.02%     
==========================================
  Files         129      129              
  Lines       25184    25312     +128     
==========================================
+ Hits        21958    22077     +119     
- Misses       3226     3235       +9     

Files Changed	Coverage Δ
include/bpf2c.h	100.00% <ø> (ø)
tests/sample/map_in_map_btf.c	100.00% <ø> (ø)
tests/sample/tail_call_sequential.c	96.96% <ø> (ø)
tools/bpf2c/bpf_code_generator.h	93.54% <ø> (ø)
libs/execution_context/ebpf_native.c	88.00% <77.02%> (-0.98%)	⬇️
libs/execution_context/ebpf_core.c	90.79% <90.00%> (-0.01%)	⬇️
tools/bpf2c/bpf_code_generator.cpp	95.87% <94.04%> (-0.21%)	⬇️
tests/sample/tail_call_recursive.c	100.00% <100.00%> (ø)
tests/unit/libbpf_test.cpp	99.94% <100.00%> (-0.01%)	⬇️

... and 16 files with indirect coverage changes

[dthaler]

Ok other than a grammar nit in a comment

[shpalani]

1. For my understanding, what is the 'recurse' type in the .values array?

Is it a function pointer? Or if it is an enum, what are the various values it can take? In short, trying to understand how the value is consumed.

2. Is the .values an optional array?

[Alan-Jowett]

1. For my understanding, what is the 'recurse' type in the .values array?

Is it a function pointer? Or if it is an enum, what are the various values it can take? In short, trying to understand how the value is consumed.

2. Is the .values an optional array?

The "values" field is defined by BPF convention as a BTF map field. It is how the types of values stored in a BPF map declared via BTF. Optionally it can also be used to declare the initial values for the BPF map that will be set when the map is created.

If you take a look at the "tests/sample/tail_call_sequential.c" sample, it should be clearer that the .values field contains the list of programs to be inserted into the BPF map. Likewise, the "tests/sample/map_in_map_btf.c" there is an example of initializing a map in map with an initial value.

[saxena-anurag]

The test case exercising this sample program (TEST_CASE("sequential_tail_call", "[libbpf]")) is overwriting the map entries manually. I think we should:

1. Update the test case to not do so, else the new code path will not be tested.
2. Instead of updating this sample, have a new sample program so that one can test normal map creation, and other can test creation + static initialization.

[Alan-Jowett]

The test was updated to not initialize the map and instead rely on the statically initialized values.

[shpalani]

From the above, with this new field,
__array(values, int(struct xdp_md* ctx));

1. Do the user program need to do bpf_map_update_elem for each tail call program?
   Eg, for (...) {
   bpf_map_update_elem(map_fd, &key, &program_fd, 0);
   }

2. If yes, What happens if the map elements are not done in that order (index), matching the .values array?