Skip to content

Conversation

@xingbowu
Copy link
Contributor

No description provided.

*
* @return oss client.
*/
OSS oss();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this method name is clear. How about newClient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good. I will make it.

* different endpoints. For more information, see:
* https://www.alibabacloud.com/help/doc-detail/31837.htm
*/
public static final String OSS_ENDPOINT = "oss.endpoint";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally try to avoid nesting in places that have a small set of specific properties. Is it possible to make these shorter and more standard?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we will introduce the aliyun catalog services named DLF in future, the DLF services also has its endpoint. we need to prefix oss. and dlf. to distingush it's an oss endpoint or dlf endpoint. That's why we use oss.endpoint here.

@jackye1995 jackye1995 added this to the Iceberg 0.13.0 Release milestone Nov 18, 2021
@rdblue
Copy link
Contributor

rdblue commented Nov 18, 2021

Looks good to me. Since I don't see an approval from @openinx, I'll wait and let him merge this when he is ready.

* If set, all Aliyun clients will be initialized by the specified factory.
* If not set, {@link DefaultAliyunClientFactory} is used as default factory.
*/
public static final String CLIENT_FACTORY = "client.factory-impl";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if they were the same, but I like that this conforms to the -impl convention that we use for dynamically loaded classes elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can make another PR to line up the behavior of aws part as "client.factory-impl"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, that works for me as well

import java.util.Map;
import org.apache.iceberg.relocated.com.google.common.base.Preconditions;

public class DefaultAliyunClientFactory implements AliyunClientFactory {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we were writing AwsClientFactory we had AwsClientFactories to vend a default client factory implementation, and that was because it seems like Iceberg does not expose the default implementation as a public class for other classes like LocationProvider, etc. I wonder if we should follow the same pattern here.

https://github.com/apache/iceberg/blob/master/aws/src/main/java/org/apache/iceberg/aws/AwsClientFactories.java

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree. As we will also introduce many different factories for aliyun clients, it's good to have a proxy to delegate to create the factories.

*
* @return oss client.
*/
OSS newClient();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to name it as ossClient because I think we will introduce another aliyun client, such as dlf client etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How aboutnewOSSClient then?

@openinx
Copy link
Member

openinx commented Nov 19, 2021

> Task :iceberg-arrow:compileScala NO-SOURCE
> Task :iceberg-arrow:classes
> Task :iceberg-arrow:jar
> Task :iceberg-arrow:sourceJar
Error: eckstyle] [ERROR] /home/runner/work/iceberg/iceberg/aliyun/src/test/java/org/apache/iceberg/aliyun/TestAliyunClientFactories.java:24:1: Extra separation in import group before 'java.util.HashMap' [ImportOrder]
Error: eckstyle] [ERROR] /home/runner/work/iceberg/iceberg/aliyun/src/test/java/org/apache/iceberg/aliyun/TestAliyunClientFactories.java:24:8: Unused import - java.util.HashMap. [UnusedImports]
Error: eckstyle] [ERROR] /home/runner/work/iceberg/iceberg/aliyun/src/test/java/org/apache/iceberg/aliyun/TestAliyunClientFactories.java:52:5: 'CTOR_DEF' should be separated from previous line. [EmptyLineSeparator]

> Task :iceberg-aliyun:checkstyleTest

> Task :iceberg-aliyun:checkstyleTest FAILED
Note: Some input files use or override a deprecated API.

Note: Recompile with -Xlint:deprecation for details.
> Task :iceberg-core:compileTestJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

131 actionable tasks: 131 executed
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':iceberg-aliyun:checkstyleTest'.
> Checkstyle rule violations were found. See the report at: file:///home/runner/work/iceberg/iceberg/aliyun/build/reports/checkstyle/test.html
  Checkstyle files with violations: 1
  Checkstyle violations by severity: [error:3]

@xingbowu , Could you help to fix the checkstyles ?

Copy link
Contributor

@jackye1995 jackye1995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks for addressing the comments!

*
* @return oss client.
*/
OSS ossClient();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this newOSSClient? We typically prefix factory method names with new.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rework and follow up this guide

Copy link
Member

@openinx openinx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now ! Let's get this merged, Thanks @xingbowu for contribution and thanks @jackye1995 and @rdblue for the careful reviewing !

@openinx openinx merged commit 0569159 into apache:master Nov 22, 2021
Initial-neko pushed a commit to Initial-neko/iceberg that referenced this pull request Nov 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants