feat: AE-1102 Add support for new serverless runtime #85
Conversation
pandyamarut
force-pushed
the
ae-1102-runt-2
branch
from
083dc6b to
aa77d86
Compare
…leanresource-based architecture
Signed-off-by: pandyamarut <[email protected]>
Signed-off-by: pandyamarut <[email protected]>
jhcipar
left a comment
jhcipar
left a comment
There was a problem hiding this comment.
left some comments - hopefully they're helpful/useful!
|
|
||
| log.info(f"LoadBalancerSls initialized with endpoint: {self.endpoint_url}") | ||
|
|
||
| async def _get_session(self) -> aiohttp.ClientSession: |
There was a problem hiding this comment.
could we reuse the existing client implementation for some of this logic?
| # LoadBalancerSls execution (Load Balancer mode) | ||
| @remote( | ||
| resource_config=my_resource_config, | ||
| type="LB", |
There was a problem hiding this comment.
Do we need the separate type attr if we're already detecting resources as load balancer sls resources?
| LoadBalancerSls client for dual-capability remote execution. | ||
| Usage: | ||
| # After manually deploying LoadBalancerSls container |
There was a problem hiding this comment.
So we always initialize a LB from a manually deployed endpoint? is there a way we can deploy via Tetra, like with the queue-based sls endpoints? It would be nice for everything to happen in an IDE, but I also have not tried out LB usage so I'm not as familiar with potential limitations there!
|
|
||
| # For now, use the hardcoded URL as requested | ||
| # TODO: Integrate with resource_config to get actual deployed endpoint URL | ||
| endpoint_url = "https://9ttr6h4l3f17w3.api.runpod.ai" |
There was a problem hiding this comment.
TODO here - just in case you're not aware!
There was a problem hiding this comment.
Yeah, the deployment part has been added in an another PR.
This PR introduces DeploymentRuntime, a new serverless runtime that enables deploying Python classes as both HTTP-accessible REST APIs and remote execution targets. Classes decorated with @Remote(type="LB") are automatically deployed as Load Balancer endpoints, making methods marked with @endpoint callable via curl/HTTP while
preserving traditional async remote execution for other methods.