[CI] Fix GPU memory leak when RemoteOpenAIServer fails to start in __init__

[AndreasKaratzas]

• When RemoteOpenAIServer.__init__ raises (e.g. health check timeout), __exit__ is never called by Python's with statement, leaking the server + EngineCore subprocesses and their GPU memory. Every subsequent test then OOMs.
• Extracted cleanup into _shutdown() called from both __exit__ and __init__'s exception handler.
• Replaced _kill_orphaned_children (psutil parent-child lookup, broken after parent is reaped) with _kill_process_group_survivors that scans /proc/*/stat for pgid members -- works even after the parent process is gone.
• Removed the Phase 2 "stabilization" fallback in _wait_for_gpu_memory_release that silently proceeded when GPU memory was still held. Now raises RuntimeError so the leaking test fails instead of poisoning all later tests.
• Increased server startup timeout from 360s to 480s (model load alone took ~350s on MI250 ROCm CI).
• Increased memory release timeout from 60s to 120s for ROCm GPU driver reclaim latency.

Test plan

• pytest -s -v models/language/pooling -m 'not core_model'

cc @kenroche

[gemini-code-assist]

Code Review

This pull request introduces several improvements to the test server cleanup logic to prevent resource leaks, particularly GPU memory, when the server fails to start. The main changes include:

1. Refactoring the cleanup logic into a _shutdown method, which is now correctly called from __init__ on startup failure.
2. Replacing the psutil-based process cleanup with a more robust implementation that directly parses /proc to find and terminate processes in a process group.
3. Making the GPU memory release check stricter by removing the "stabilization" fallback, ensuring that tests with memory leaks fail immediately.
4. Increasing server startup and memory release timeouts to accommodate slower CI environments.

The changes are well-reasoned and significantly improve the reliability of the test suite. I have one suggestion to make the new process cleanup logic even more robust.

[gemini-code-assist]

The current parsing of /proc/<pid>/stat using stat.split() is not robust against process names containing spaces. The process name (field 2) is enclosed in parentheses and can contain spaces. A simple split() will incorrectly parse the fields if the name has spaces, causing the check to fail and potentially leaving orphan processes running.

A more robust approach is to find the last parenthesis of the process name and split the rest of the string. This ensures the correct fields are extracted regardless of the process name's content. This change also removes the now-incorrect comment about the field index.

                # The process name is in parentheses. Find the last ')'
                # to locate the end of the process name.
                last_paren_idx = stat.rfind(')')
                if last_paren_idx == -1:
                    continue

                # The fields after the process name start after ') '.
                # pgrp is the 3rd field after the process name (state, ppid, pgrp).
                fields = stat[last_paren_idx + 2:].split()
                if len(fields) > 2 and int(fields[2]) == pgid:
                    members.append(int(entry.name))

[AndreasKaratzas]

Done :) Replaced with os.getpgid which is cleaner too.