Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amazon S3のendpointの指定について #4

Closed
ume3 opened this issue Nov 21, 2011 · 2 comments
Closed

Amazon S3のendpointの指定について #4

ume3 opened this issue Nov 21, 2011 · 2 comments

Comments

@ume3
Copy link

ume3 commented Nov 21, 2011

はじめまして。
1週間前ほどからfluent-plugin-s3に興味をもって検証させていただいているものです。
報告させていただきます。
※ 不慣れなもので、記述場所にあやまりがあれば、消去のほどよろしくお願いします。

導入予定かもしれませんが、
・s3_endpointの指定
についてご確認させていただきます。

こちらは、aws-sdkにて記述を検討が前提なのか、
それとも、下記のように設定ファイルに埋め込むことを予定されていますでしょうか?

バージョン:fluent-plugin-s3 (0.2.1)
対象ファイル:https://github.com/fluent/fluent-plugin-s3/blob/master/lib/fluent/plugin/out_s3.rb

■ end_point差し込みのイメージ



# diff -u /tmp/fluent-plugin-s3/lib/fluent/plugin/out_s3.rb /tmp/fluent-plugin-s3/lib/fluent/plugin/out_s3a.rb
--- /tmp/fluent-plugin-s3/lib/fluent/plugin/out_s3.rb   2011-11-21 15:03:09.474556318 +0900
+++ /tmp/fluent-plugin-s3/lib/fluent/plugin/out_s3a.rb  2011-11-21 15:04:41.110590881 +0900
@@ -18,6 +18,7 @@
   config_param :aws_key_id, :string
   config_param :aws_sec_key, :string
   config_param :s3_bucket, :string
+  config_param :s3_endpoint, :string

   def configure(conf)
     super
@@ -28,6 +29,7 @@
   def start
     super
     @s3 = AWS::S3.new(
+      :s3_endpoint=>@s3_endpoint,
       :access_key_id=>@aws_key_id,
       :secret_access_key=>@aws_sec_key)
     @bucket = @s3.buckets[@s3_bucket]

■ 設定ファイル例


<match pattern>
  type s3

  aws_key_id YOUR_AWS_KEY_ID
  aws_sec_key YOUR_AWS_SECRET/KEY
  s3_bucket YOUR_S3_BUCKET_NAME
  path logs/
  buffer_path /var/log/fluent/s3

  ## Tokyo
  s3_endpoint s3-ap-northeast-1.amazonaws.com

  time_slice_format %Y%m%d-%H
  time_slice_wait 10m
  utc
</match>

ご指摘のほどよろしくお願いします。

@hotchpotch
Copy link
Member

報告ありがとうございます。
s3_endpoint に対応しました。

@ume3
Copy link
Author

ume3 commented Dec 19, 2011

対応ありがとうございます。
こちらの設定ファイルで一括して管理できるので助かります。

つたない要望にこたえていただき、ありがとうございました。

chillaxd pushed a commit to chillaxd/fluent-plugin-s3 that referenced this issue Dec 19, 2016
# This is the 1st commit message:

# This is a combination of 3 commits.
# This is the 1st commit message:
# This is a combination of 3 commits.
# This is the 1st commit message:
Out S3 change

so that user can use minimal policy at AWS, to work with fluentd
like S3:PutObject

# This is the commit message fluent#2:

Spacing handled in out-S3 plugin

# This is the commit message fluent#3:

Identation modified

# This is the commit message fluent#4:

Introduce check_bucket paramater
To skip the using of auto_create_bucket for bucket existence checking

# This is the commit message fluent#5:

Fixes:

Test cases added.
README has been modified with check_bucket's usage and sample AWS policy.

# This is the commit message fluent#6:

Fixes:

Issues for extra spoaces. Modified

# This is the commit message fluent#7:

Spacing issues

# This is the commit message fluent#8:

Update test_out_s3.rb

# This is the commit message fluent#9:

time slicer modified for utc also

# This is the commit message fluent#10:

removing test cases

# This is the commit message fluent#1:

renaming time_slicer variable

# This is the commit message fluent#2:

Test casees added

Test cases for out_s3 plugin has been added. This feature will work with hardened
S3 policy like with only PutObject permission. As it will not check bucket existence
not even object existence, as for each object creation it will name it with timestamp.
So anyhow it will be unique everytime.

Only thing it willl assume that bucket is created already.

# This is the commit message fluent#3:

Some variable renamed

# This is the commit message fluent#2:

Clean up dependencies

# This is the commit message fluent#3:

Update README for placeholders

# This is the commit message fluent#2:

v0.8.0.rc1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants