[SPARK-41705][CONNECT] Move generate_protos.sh to dev/

[tedyu]

What changes were proposed in this pull request?

This PR moves generate_protos.sh from connector/connect/dev to dev directory.

Why are the changes needed?

connector/connect/dev only contains one script. Moving generate_protos.sh to dev follows practice for other scripts.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing test suite.

[tedyu]

cc @HyukjinKwon

[tedyu]

KubernetesSuite.scala failure may not be related to the PR.

pushd /__w/connector/connect/common/src/main
/__w/spark/spark/dev/generate_protos.sh: line 39: pushd: /__w/connector/connect/common/src/main: No such file or directory

I tried to account for SPARK_HOME:

pushd ${SPARK_HOME}/connector/connect/common/src/main

Though the above doesn't seem to work.

[grundprinzip]

I don't think it's worth moving it atm. It has a clear relationship to the contained code in the connect package.

[HyukjinKwon]

Actually maybe now it's time to consider moving these (including README.md) together since we're going to package them together (#38477). The reason is that all dev related scripts are located there e.g., dev/reformat-python, etc.

[HyukjinKwon]

Let's also fix dev/check-codegen-python.py script together to fix up the linter.

[HyukjinKwon]

Should also better file a JIRA.

[HyukjinKwon]

Since we already did cd "$SPARK_HOME" above, I guess we won't need this change anyway (?)

[zhengruifeng]

the code gen checker failed:

Traceback (most recent call last):
  File "./dev/check-codegen-python.py", line 84, in <module>
    check_connect_protos()
  File "./dev/check-codegen-python.py", line 49, in check_connect_protos
    run_cmd(f"{SPARK_HOME}/dev/generate_protos.sh {tmp}")
  File "./dev/check-codegen-python.py", line 43, in run_cmd
    return subprocess.check_output(cmd.split(" ")).decode("utf-8")
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/__w/spark/spark/dev/generate_protos.sh', '/tmp/tmpdk4qg3nj']' returned non-zero exit status 1.

[HyukjinKwon]

will take this over with moving related README.md together.

[HyukjinKwon]

#39338. I will add you as a co-author there.