[*WIP*] Controller - Agent interface

[nikhil-zlai]

Summary

hodge podge of necessary changes - will cleanup in a bit

1. thrift interface that the agent is going to work with - "yarn job" abstracts over spark & flink jobs
2. jobs are very general - takes cli args and env and chronon version - nothing more nothing less
3. bazel fixes - shopt isn't available on zsh, and many other issues with intellij bazel sync command
4. WIP - thrift router - generate type web client from router spec.

Checklist

• Added Unit Tests
• Covered by existing CI
• Integration tested
• Documentation update

Summary by CodeRabbit

• New Features
  • Introduced an enhanced job management system that supports dynamic resource scaling, advanced job tracking, and robust orchestration of transient job clusters.
  • Enabled improved communication of job status and resource usage, offering more effective tracking for job execution.
  • Added new data structures and enumerations for managing job execution and resource allocation.
• Refactor
  • Streamlined response formatting and error handling, resulting in more consistent and reliable outputs for client interactions.
  • Improved code readability through formatting adjustments in test cases and method signatures.

[coderabbitai]

Actionable comments posted: 0

♻️ Duplicate comments (2)

service_commons/src/main/java/ai/chronon/service/RouteHandlerWrapper.java (2)

240-262: 🛠️ Refactor suggestion

Add input validation and field limit.

Previous null check concern still applies.

Add field limit to prevent excessive processing:

 public static Map<String, String> convertPojoToMap(Object pojo) {
+    if (pojo == null) {
+        throw new IllegalArgumentException("Input POJO cannot be null");
+    }
     Map<String, String> result = new HashMap<>();
     Class<?> pojoClass = pojo.getClass();
 
     // Get all getters
     Map<String, Method> getters = Arrays.stream(pojoClass.getMethods())
             .filter(RouteHandlerWrapper::isGetter)
+            .limit(100) // Prevent excessive field processing
             .collect(Collectors.toMap(RouteHandlerWrapper::getFieldNameFromGetter, method -> method));

---

288-333: 🛠️ Refactor suggestion

Add size limits and handle nested structures.

Previous map size limit concern still applies.

Add list size limit and handle nested structures:

     // Handle Lists
     if (List.class.isAssignableFrom(valueClass)) {
+        if (((List<?>) value).size() > 1000) {
+            throw new IllegalArgumentException("List size exceeds maximum limit of 1000 entries");
+        }
         return ((List<?>) value).stream()
                 .map(RouteHandlerWrapper::convertToString)
+                .limit(1000)
                 .collect(Collectors.joining(","));
     }

🧹 Nitpick comments (1)

service_commons/src/main/java/ai/chronon/service/RouteHandlerWrapper.java (1)

273-286: Add method name validation.

Add prefix validation to prevent processing invalid method names.

 private static String getFieldNameFromGetter(Method method) {
     String methodName = method.getName();
+    if (!methodName.startsWith("get") && !methodName.startsWith("is")) {
+        throw new IllegalArgumentException("Method name must start with 'get' or 'is': " + methodName);
+    }
     String fieldName;

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between 27176c1 and c0b7284.

📒 Files selected for processing (2)

• .bazelrc.local (1 hunks)
• service_commons/src/main/java/ai/chronon/service/RouteHandlerWrapper.java (1 hunks)

✅ Files skipped from review due to trivial changes (1)

• .bazelrc.local

⏰ Context from checks skipped due to timeout of 90000ms (4)

• GitHub Check: non_spark_tests
• GitHub Check: scala_compile_fmt_fix
• GitHub Check: enforce_triggered_workflows
• GitHub Check: build-and-push

🔇 Additional comments (1)

service_commons/src/main/java/ai/chronon/service/RouteHandlerWrapper.java (1)

264-271: LGTM!

Correctly implements Java Bean getter validation.

[nikhil-zlai]

@kumar-zlai can you take this over - we just want the agent code pushed in - please kill everything else.