Incorrect memory pool behavior when out of memory/nullptr passed

[knkemu]

sceKernelMemoryPoolExpand can easily run into undefined behavior due to dereference of std::map::end when out of memory

shadPS4/src/core/memory.cpp

Lines 68 to 84 in fe389e5

	PAddr MemoryManager::PoolExpand(PAddr search_start, PAddr search_end, size_t size, u64 alignment) {
	std::scoped_lock lk{mutex};
	auto dmem_area = FindDmemArea(search_start);
	const auto is_suitable = [&] {
	const auto aligned_base = alignment > 0 ? Common::AlignUp(dmem_area->second.base, alignment)
	: dmem_area->second.base;
	const auto alignment_size = aligned_base - dmem_area->second.base;
	const auto remaining_size =
	dmem_area->second.size >= alignment_size ? dmem_area->second.size - alignment_size : 0;
	return dmem_area->second.is_free && remaining_size >= size;
	};
	while (!is_suitable() && dmem_area->second.GetEnd() <= search_end) {
	dmem_area++;
	}
	ASSERT_MSG(is_suitable(), "Unable to find free direct memory area: size = {:#x}", size);

Some games can expect to run out of memory and take measures, but shadPS4 currently just runs into undefined behavior and crashes.
I think there's even more examples within that file so I think it should be slightly reworked to handled out of memory scenarios by returning ENOMEM

Furthermore, the following check is present on sceKernelMemoryPoolReserve:

shadPS4/src/core/libraries/kernel/memory_management.cpp

Lines 382 to 389 in fe389e5

	s32 PS4_SYSV_ABI sceKernelMemoryPoolReserve(void* addrIn, size_t len, size_t alignment, int flags,
	void** addrOut) {
	LOG_INFO(Kernel_Vmm, "addrIn = {}, len = {:#x}, alignment = {:#x}, flags = {:#x}",
	fmt::ptr(addrIn), len, alignment, flags);
	if (addrIn == nullptr) {
	LOG_ERROR(Kernel_Vmm, "Address is invalid!");
	return SCE_KERNEL_ERROR_EINVAL;

However, CUSA49295 calls sceKernelMemoryPoolReserve with the first argument as NULL hardcoded - which suggests that recognizing addrIn == nullptr as an invalid parameter is wrong.