Internet gateway delete logic is flawed

[rcgoodfellow]

Internet gateway (IG) deletion has two modes, cascade and no cascade. The no cascade option should fail if there are routes pointing at the IG. The no cascade option should delete an internet gateway and its associated IP pool / address attachments. However in the code below we bail if there are IP pool / address associations and we don't even check routes.

omicron/nexus/db-queries/src/db/datastore/vpc.rs

Lines 1645 to 1686 in 83a881b

	pub async fn vpc_delete_internet_gateway_no_cascade(
	&self,
	opctx: &OpContext,
	authz_igw: &authz::InternetGateway,
	) -> DeleteResult {
	opctx.authorize(authz::Action::Delete, authz_igw).await?;
	let conn = self.pool_connection_authorized(opctx).await?;
	let err = OptionalError::new();
	#[derive(Debug)]
	enum DeleteError {
	IpPoolsExist,
	IpAddressesExist,
	}
	self.transaction_retry_wrapper("vpc_delete_internet_gateway_no_cascade")
	.transaction(&conn, |conn| {
	let err = err.clone();
	async move {
	// Delete ip pool associations
	use nexus_db_schema::schema::internet_gateway_ip_pool::dsl as pool;
	let count = pool::internet_gateway_ip_pool
	.filter(pool::time_deleted.is_null())
	.filter(pool::internet_gateway_id.eq(authz_igw.id()))
	.count()
	.first_async::<i64>(&conn)
	.await?;
	if count > 0 {
	return Err(err.bail(DeleteError::IpPoolsExist));
	}
	// Delete ip address associations
	use nexus_db_schema::schema::internet_gateway_ip_address::dsl as addr;
	let count = addr::internet_gateway_ip_address
	.filter(addr::time_deleted.is_null())
	.filter(addr::internet_gateway_id.eq(authz_igw.id()))
	.count()
	.first_async::<i64>(&conn)
	.await?;
	if count > 0 {
	return Err(err.bail(DeleteError::IpAddressesExist));
	}

[ijsnow]

Apologies if I stepped on anyone's toes, but I opened a PR for this: #11016