Proposal: add llgo:tls and llgo:gls variable directives

[cpunion]

Summary

Add two package-variable directives:

//llgo:tls
var threadState T

//llgo:gls
var goroutineState U

They preserve ordinary Go variable syntax: reads, writes, address-taking,
closures, and atomics use the current owner's stable address. TLS and GLS are
separate source and initializer kinds. The current one-pthread-per-goroutine
runtime maps both to the same physical backend; a future scheduler may move GLS
to goroutine context storage without changing source declarations.

Locality variables must be package-level and unexported. They cannot use
go:linkname, go:embed, or the blank identifier. Cross-package use therefore
goes through ordinary exported functions compiled in the package that owns the
local variable.

Package boundaries

The implementation keeps directive semantics separate from lowering:

• internal/directive only parses and normalizes directives.
• internal/locality validates TLS/GLS declarations and prepares initializer
  helpers.
• internal/locality/layout is a pure type/layout planner with no SSA, LLVM,
  target, build-cache, or runtime dependency.
• ssa stores program-wide locality metadata and emits local-context/accessor
  code.
• cl/locality_lower.go integrates the plan into normal variable lowering.
• internal/build preloads source metadata before cached package compilation
  and fingerprints whether entry-context instrumentation is required.
• runtime/internal/runtime owns allocation, GC reachability, and teardown.

No target name or new build tag is inspected by the compiler. When native TLS
is selected, LLVM TLS is emitted. Bare-metal runtime files continue to choose
their runtime implementation through existing source constraints; the language
layer does not reject TLS/GLS declarations for a target.

Compilation and link flow

For every package, before function bodies are lowered:

1. The source prepass records directives and go:linkname metadata, including
   packages whose compiled module may come from cache.
2. internal/locality creates explicit variable-initializer helpers while
   preserving Go initialization order.
3. layout.Plan classifies each variable as native TLS or a field in the
   package's GC-visible payload.
4. The package module defines its native TLS globals, package cache/accessor,
   initializer guards, dispatcher, and ensure functions.
5. Function lowering uses fixed field indexes from the plan. Package bases and
   successful initializer checks are reused along SSA dominator paths.

A package with a GC-visible payload defines this pair:

p.__llgo_local_cache  native TLS uintptr
p.__llgo_local_block  () *pPayload, alwaysinline

The cache is package-owned. Because p's locality variables are unexported,
another package calls an exported function in p; that function uses the same
p.__llgo_local_cache. Importers do not define or directly access another copy
of p's local storage.

After package modules are built or loaded:

• The generated main entry reserves a candidate LocalContext, enters it before
  package initialization, and leaves it after main.main returns.
• Every generated pthread wrapper for go f(...) reserves and enters a fresh
  context before invoking the Go call.
• C-exported Go entries use the same enter/leave sequence. A nested entry reuses
  an already active context.
• Runtime, package, and main modules are linked normally.

flowchart TD
    A[Load source and type information] --> B[Parse directives and linknames]
    B --> C[Prepare initializer helpers]
    C --> D[Plan native TLS and package payloads]
    D --> E[Compile or load package modules]
    E --> F[Emit package caches, accessors, guards, dispatchers]
    D --> G{Program needs LocalContext?}
    G --> H[Instrument main entry]
    G --> I[Instrument goroutine wrappers]
    G --> J[Instrument C-exported entries]
    F --> K[Link packages, runtime, and main]
    H --> K
    I --> K
    J --> K

Loading

Storage selection

Variable type	Current TLS backend	Current GLS backend
Pointer-free	native LLVM thread_local	native LLVM thread_local
GC-visible	field in one lazy package payload	field in the same package payload

Pointer-bearing TLS and GLS fields in one package share one payload. Their
initializer guards remain separate by locality kind.

Runtime memory layout

Example package:

//llgo:tls
var count uint64

//llgo:gls
var current *Node

//llgo:tls
var state struct {
    head *Entry
    n    int
}

On a 64-bit hosted target, one active owner has this layout:

native TLS for package p
  p.count                 uint64       8 B
  p.__llgo_local_cache    uintptr      8 B, non-root payload cache
  p.__llgo_tls_init$guard uint8        only with explicit TLS initializers
  p.__llgo_gls_init$guard uint8        only with explicit GLS initializers

outer Go entry stack
  LocalContext
    blocks -------------------------------+
                                           |
GC allocation created when p is touched   |
  [leading alignment padding]             |
  [next payload pointer              8 B]  |
  [cacheSlot -> &p.__llgo_local_cache 8 B] |
  [aligned p payload] <--------------------+
      current *Node                   8 B
      state.head *Entry               8 B
      state.n int                     8 B

LocalContext.blocks and each header's next field hold payload addresses.
The stack pointer is the GC root. The TLS cache is a uintptr and intentionally
is not a GC root.

The list is allocation/root order only. It is not an LRU and is never searched
on access. For example, after touching packages p, q, and r:

ctx.blocks -> r.payload -> q.payload -> p.payload -> nil
p.__llgo_local_cache -> p.payload
q.__llgo_local_cache -> q.payload
r.__llgo_local_cache -> r.payload

An access to p loads p.__llgo_local_cache directly; it does not walk
r -> q -> p and does not reorder the list. The list exists only so the GC can
reach every active block and so owner teardown can clear every cache slot.
Initializer failure values use a separate failure cache/block only after a
panic.

Runtime access flow

The always-inline package accessor is conceptually:

payload = load_tls p.__llgo_local_cache
if payload != 0:
    return payload
return runtime.LocalPackage(&p.__llgo_local_cache, size, align)

On first touch, LocalPackage:

1. verifies that an outer Go entry context is active;
2. allocates one zeroed, aligned header plus package payload;
3. prepends it to LocalContext.blocks for GC reachability;
4. writes the payload address to the package TLS cache; and
5. returns the payload.

Every later uncached function path performs one O(1) TLS cache check. Within a
function, the compiler can reuse a package base along dominated paths and omit
additional checks. There is no hash lookup, list search, lock, locality
pthread.Key, or per-variable root registration.

At owner exit, the runtime walks the root list once, clears each recorded TLS
cache slot, and severs next and cacheSlot links. It does not explicitly free
a block: an escaped address can keep that block alive, while cleared links stop
it from retaining unrelated blocks.

sequenceDiagram
    participant C as Compiled access in package p
    participant A as p.__llgo_local_block
    participant T as p TLS cache
    participant R as runtime.LocalPackage
    participant X as LocalContext root list
    C->>A: request p payload
    A->>T: load p.__llgo_local_cache
    alt cache hit
        T-->>A: p payload
        A-->>C: typed payload
    else first touch
        A->>R: cache address, size, alignment
        R->>X: allocate and prepend rooted block
        R->>T: cache p payload
        R-->>A: p payload
        A-->>C: typed payload
    end
    C->>C: add fixed field offset

Loading

Main and goroutine execution

Initial main owner

1. The generated executable entry installs the main LocalContext before Go
   package initialization.
2. Each package marks its local initializer guards ready for the initial owner.
3. Normal Go package initialization executes the original variable
   initializers once in dependency/order semantics.
4. Pointer-free writes go to native TLS. GC-visible writes obtain the package
   payload and store at their fixed field offsets.
5. main.main uses the same context and sees those initialized values.

New goroutine owner

1. The pthread wrapper starts with zeroed native TLS and an empty
   LocalContext.
2. It does not enumerate packages, allocate package blocks, or replay
   initializers eagerly.
3. The first access to package p processes only p at that point. If code in
   p then calls package q, q is handled independently when accessed.
4. Another goroutine has different TLS slots and a different context.

Payload allocation remains sparse: a goroutine that touches only p and q
allocates payloads only for those packages. The direct cache itself costs one
fixed TLS word per package that has a GC-visible payload.

Separate outer calls from the same foreign pthread are a documented boundary:
this proposal does not promise package-payload persistence across those outer
entries.

Initializer semantics

//llgo:gls
var a = 1

//llgo:gls
var b = makeB()

//llgo:gls
var c, d = pair()

//llgo:gls
var x T

For each new owner:

• explicit initializers are replayed lazily when that package/locality kind is
  first accessed;
• a is assigned 1;
• makeB() is called once;
• pair() is called once for the grouped declaration;
• pure zero-value x has no helper; and
• Go func init() is not replayed and keeps normal once-per-program behavior.

One dispatcher and one native TLS guard are generated per package/locality kind,
not per variable. First access to any variable of that kind runs all explicit
helpers for the kind in Go initialization order. Recursive access observes
partial initialization. A failed initializer remains failed and re-panics with
the saved value on later accesses, including panic(nil); its any payload is
allocated only on failure.

The ready guard check is always-inline, and successful checks/package bases are
reused along dominated paths. This avoids repeated helper calls while preserving
the lazy execution point.

Cost model

64-bit condition	Cost
context-enabled outer Go entry	one 8-byte candidate stack slot; only the outer slot is the root
package with a GC-visible payload	one 8-byte native TLS cache per pthread, even if that package is untouched
untouched package payload	no GC allocation
touched package payload	16-byte header + alignment padding + payload + allocator rounding
package/kind with explicit initializers	one native TLS guard byte and one 8-byte failure-cache slot
successful initializer	no failure block
failed initializer	one lazy block containing an any value plus the normal 16-byte header
owner exit	one linear walk over blocks allocated by that owner

Programs containing only zero-initialized, pointer-free native TLS locals do not
need a LocalContext. The hot access path has no lock, heap allocation, package
list traversal, or pthread_getspecific call. Cold first-touch allocation is
not optimized by this proposal.

Measurements

Apple M4 Max, macOS arm64, Go 1.26.5, LLVM 19.1.7, real LLGo,
GOMAXPROCS=2, -count=8 -benchtime=750ms -benchmem unless noted.

Comparable imported //go:noinline func() uintptr readers:

Read path	Previous list median	Direct-cache median	Current range
ordinary global	~0.778 ns/op	0.770 ns/op	0.7655-0.7774
native LLVM TLS	1.300 ns/op	1.300 ns/op	1.295-1.307
hot GLS package payload	1.561 ns/op	1.418 ns/op	1.390-1.422

All report 0 B/op, 0 allocs/op. Direct GLS is about 0.118 ns, or 9.1%,
over native TLS in this run.

The retained pre-migration runtime benchmark records the old pthread-key cost:

Runtime access path	Earlier 1s x 5 range	Current median	Current 750ms x 8 range	Heap accounting
pthread-key GetThreadDefer	2.321-2.337 ns/op	2.323 ns/op	2.310-2.363 ns/op	0 B/op, 0 allocs/op

#2079 intentionally leaves GetThreadDefer on its existing
pthread_getspecific implementation, so this is a reproducible old-runtime
baseline for the follow-up migration. It is not the same function as the
generic reader table above; a same-function comparison belongs in the
runtime-state follow-up.

The retained alternating benchmark performs two serialized GLS reads per op.
One of its packages also has explicit GLS initializers, so that read checks the
shared package/kind initializer guard before reading its payload:

Working set	Previous list batch median	Direct-cache batch median	Direct-cache ns/read	Current range
alternating two packages, two reads/op	7.334 ns/op	3.510 ns/op	1.755	3.490-3.543 ns/op

This is an apples-to-apples backend regression benchmark: direct caches reduce
the same guarded workload by 52.1%. It is not a pure two-read latency estimate.
The zero-initializer independent-package benchmark below is the scaling check.

Each independent working-set op is one batch containing N independent
Read calls, so batch ns/op is expected to grow with N:

Batch size (reads/op)	Median batch ns/op	Throughput ns/read	Heap accounting
2	2.656	1.328	0 B/op, 0 allocs/op
4	5.161	1.290	0 B/op, 0 allocs/op
8	9.937	1.242	0 B/op, 0 allocs/op

The per-read throughput does not increase as the working set grows from 2 to 8
packages, confirming that hot access no longer depends on touched-package
count. The slight decrease comes from independent calls overlapping; this
table measures throughput scaling, not single-read latency below the 1.418
ns/op result.

Fixed-count goroutine measurements also show no space regression: entry is
1552-1560 B/op versus the previous 1570-1572 B/op, and first package touch is
1583-1591 B/op versus 1592-1606 B/op. These counters are allocator/run-sensitive;
the relevant conclusion is that the direct cache did not add measured per-entry
allocation.

Final macOS arm64 disassembly contains no __llgo_local_block symbol after
optimization. The hit sequence is inlined into ReadGLSPackage; only the zero
cache branch calls runtime.LocalPackage.

Validation

• macOS arm64, Go 1.26.5, LLVM 19.1.7: full ssa, focused compiler locality,
  codegen FileCheck, test/llgoext, and test/llgoext/localitymulti pass.
• Ubuntu ARM64 container, Go 1.24.2, LLVM 19.1.7, 2 CPU and 15 GiB limit: both
  real-LLGo test packages pass.
• Ubuntu AMD64 container: focused internal/locality/..., ssa, and cl pass;
  the root real-LLGo locality suite passes. QEMU itself was unstable while
  compiling the separate multi-package suite, so native AMD64 CI remains the
  authoritative run for that architecture.
• internal/locality and internal/locality/layout are each 100% covered.
• All 63 instrumented changed production lines in the root Go module are hit,
  with zero uncovered or partial lines. Runtime is a nested module and is
  covered by the real-LLGo integration tests.
• No existing test is skipped or ignored.

Implementation: #2079. The GLS runtime-state consumer remains a follow-up after
this facility is green.