[airflow] Third positional parameter not named ti_key should be flagged for BaseOperatorLink.get_link (AIR303)#22828
Conversation
| #[derive(Clone, Debug, Eq, PartialEq)] | ||
| pub(crate) enum FunctionSignatureChange { | ||
| /// Carries a message describing the function signature change. | ||
| Message(&'static str), | ||
| } | ||
|
|
There was a problem hiding this comment.
The original definition in function_signature_change_in_3.rs becomes redundant as these enum items only encapsulate a message. Therefore, combine these into a single Message item, and move it to helpers.rs. This enum will be dedicated used for AIR303, or function signature change in future version of Airflow.
#[derive(Clone, Debug, Eq, PartialEq)]
pub(crate) enum FunctionSignatureChangeType {
/// Function signature changed to only accept keyword arguments.
KeywordOnly { message: &'static str },
/// Function signature changed to not accept certain positional arguments.
PositionalArgumentChange { message: &'static str },
}There was a problem hiding this comment.
I'm totally fine with this change if y'all prefer it, but I think it would also be fine to keep the old version if you think you'll want to differentiate between the variants later. You could always add a message method to avoid having to destructure the enum everywhere you use it. For example:
impl FunctionSignatureChangeType {
fn message(&self) -> &'static str {
match self {
Self::KeywordOnly { message } | Self::PositionalArgumentChange { message } => message,
}
}
}Alternatively, I think you could just use a &'static str for the message field instead of having this enum type at all?
|
|
||
| /// This is a helper function to check if the given function definition is a method | ||
| /// that inherits from a base class. | ||
| pub(crate) fn is_method_in_subclass<F>( |
There was a problem hiding this comment.
This helper function generalizes the function is_execute_method_inherits_from_airflow_operator defined in the removal_in_3.rs. The function is checking if a subclass of airflow's BaseOperator implements a execute function. In AIR303, we need to check if a subclass of airflow's BaseOperatorLink implements a get_link function. Therefore, this is something reusable across multiple rules. I parameterized the method name and the match condition, to reduce redundant definition for similar checks.
There was a problem hiding this comment.
Nice, I like this! One nit, it might make sense to have the Fn type take a &QualifiedName instead of the less type-safe &[&str]. Just an idea, though.
crates/ruff_linter/src/rules/airflow/rules/function_signature_change_in_3.rs
Show resolved
Hide resolved
crates/ruff_linter/src/rules/airflow/rules/function_signature_change_in_3.rs
Outdated
Show resolved
Hide resolved
|
2c7f7d2 to
78dccc5
Compare
| let is_valid_signature = match positional_count { | ||
| // check valid signature `def get_link(self, operator, *, ti_key)` | ||
| 2 => parameters | ||
| .kwonlyargs | ||
| .iter() | ||
| .any(|p| p.name().as_str() == "ti_key"), | ||
| // check valid signature `def get_link(self, operator, ti_key)` | ||
| 3 => parameters | ||
| .posonlyargs | ||
| .iter() | ||
| .chain(parameters.args.iter()) | ||
| .nth(2) | ||
| .is_some_and(|p| p.name().as_str() == "ti_key"), | ||
| _ => false, | ||
| }; | ||
|
|
||
| if !is_valid_signature { | ||
| checker.report_diagnostic( | ||
| Airflow3IncompatibleFunctionSignature { | ||
| function_name: "get_link".to_string(), | ||
| change: FunctionSignatureChange::Message( | ||
| "Use `def get_link(self, operator, *, ti_key)` or `def get_link(self, operator, ti_key)` as the method signature.", | ||
| ), | ||
| }, | ||
| function_def.name.range(), |
There was a problem hiding this comment.
updated the logic to check for valid signature pattern, which is either def get_link(self, operator, *, ti_key) or ``def get_link(self, operator, ti_key)`, and the diagnostic will be raised when the method signature is evaluated to invalid.
The name of the first two positional parameters doesn't matter for the rule. If there are 2 positional parameters (i.e., self, operator), check if the keyword-only parameter is named ti_key. If there are 3 positional parameters, check if the third one is named ti_key.
The message and test cases are updated accordingly.
| let positional_count = parameters.posonlyargs.len() + parameters.args.len(); | ||
|
|
||
| let is_valid_signature = match positional_count { | ||
| // check valid signature `def get_link(self, operator, *, ti_key)` |
There was a problem hiding this comment.
what would happen if we use def get_link(self, operator, ti_key)
There was a problem hiding this comment.
In this case, I think we have 3 positional parameters, and it will not match the first condition which check if the number of positional parameters is 2. It will proceed to the next condition and check if there are 3 positional parameters and iterate through it see if the 3rd one is named ti_key. I’ve added a test case for this but will also do a double check. Thanks!
1177bca to
20daae9
Compare
|
|
||
| /// This is a helper function to check if the given function definition is a method | ||
| /// that inherits from a base class. | ||
| pub(crate) fn is_method_in_subclass<F>( |
There was a problem hiding this comment.
Nice, I like this! One nit, it might make sense to have the Fn type take a &QualifiedName instead of the less type-safe &[&str]. Just an idea, though.
| #[derive(Clone, Debug, Eq, PartialEq)] | ||
| pub(crate) enum FunctionSignatureChange { | ||
| /// Carries a message describing the function signature change. | ||
| Message(&'static str), | ||
| } | ||
|
|
There was a problem hiding this comment.
I'm totally fine with this change if y'all prefer it, but I think it would also be fine to keep the old version if you think you'll want to differentiate between the variants later. You could always add a message method to avoid having to destructure the enum everywhere you use it. For example:
impl FunctionSignatureChangeType {
fn message(&self) -> &'static str {
match self {
Self::KeywordOnly { message } | Self::PositionalArgumentChange { message } => message,
}
}
}Alternatively, I think you could just use a &'static str for the message field instead of having this enum type at all?
20daae9 to
c11b27c
Compare
| F: Fn(QualifiedName) -> bool, | ||
| { | ||
| if function_def.name.as_str() != method_name { | ||
| return false; | ||
| } | ||
|
|
||
| let ScopeKind::Class(class_def) = semantic.current_scope().kind else { | ||
| return false; | ||
| }; | ||
|
|
||
| any_qualified_base_class(class_def, semantic, &is_base_class) |
There was a problem hiding this comment.
updated the helper to let Fn type take QualifiedName instead of &[&str], and use any_qualified_base_class for base class check. I got help from AI tool for it, wonder if it is the proper way to implement, still try to understand it.
Lee-W
left a comment
There was a problem hiding this comment.
Just checked the code and comments again. LGTM. Thanks!
Summary
Context:
apache/airflow#41641
apache/airflow#46415
Old Signature
New Signature
This PR extends AIR303 to detect deprecated
get_linkmethod signatures inBaseOperatorLinksubclasses, which is a method definition.BaseOperatorLinkis a base class can be extended by the user, and theget_linkmethod is called by Airflow. As the way Airflow internally call this method change, we need to flag the user's implementation if the method definition is not match with the new signature (@Lee-W , please correct me if I understand it wrong, whenever you have time, thanks). The rule detects deprecated signatures where:BaseOperatorLink(fromairflow.modelsorairflow.sdk)get_linkti_keyTest Plan
The test cases are added to
AIR303.pyand the test snapshot is updated.