Support for COS - ActivityTracker integration and IP filtering configuration

[pauljegouic]

Hi there,

We would like to be able to for each bucket to:

• Configure the activity_tracker_crn to integrate with,
• Configure the IP filtering.

though the Terraform resource.

For now, we are using a null_resource that uses COS SDK.

[pauljegouic]

Hello there, there is an additional need regarding this. Let me know if you want me to create another ticket.

• From the console I am still able to configure the Bucket AccessPolicies (that includes the IP Firewall)
• With Terraform, on the refresh phase, I got a 403 Forbidden error.

It should not be.

@hkantare

[hkantare]

    Configure the activity_tracker_crn to integrate with,
    Configure the IP filtering.

We are working on this feature to attach firewall ip's activity tracker, monitoring crn...It will be part of the next release

[pauljegouic]

Great !

[hkantare]

@ifs-pauljegouic Regarding this With Terraform, on the refresh phase, I got a 403 Forbidden error.

Since you added some Bucket AccessPolicies (that includes the IP Firewall) did you added your systemIP to the list of allowed IP's or your system IP fall in the range of IP address...Until you add it you can't access the buckets

[hkantare]

The above comments are based on our investigation we found it

[hkantare]

[pauljegouic]

ARGH so even the properties (including IDs ?) are handled on the data plane of COS ...

So we don't have any solution on this use case ?

• Provision a Bucket with AccessPolicies
• Manage those AccessPolicies from Terraform (to keep the track of configurations) from another location than the ones informed in AccessPolicies (I am performing my Terraform execution from a CICD with dynamic IPs ... :'()

I mean no easy solution, it implies SSH tunnel or VPN.

I asked the COS team to distinguish more the "administration of the service" and the "usage of the service".

[pauljegouic]

Mmmmm @hkantare I need to make more tests.

Yesterday I SWEAR, I was able to edit the access policies from the console even if I was doing this from another IP than the ones informed in "Authorized IPs"

And today, I am not able.

[pauljegouic]

Oh wow. After some investigations, I think I will have to make a ticket support ...

Here is my usecase:

1/ I provision a COS with a Bucket
2/ I inform an AccessPolicy with an IP 192.168.0.0/16 (which is not my SystemIP)
3/ I try to navigate under Configuration Bucket --> Denied OK
4/ I try to navigate under Object --> Denied OK
5/ I go back to the AccessPolicies menu, I can see the IPs allowed, and still edit them.
6/ I add one.
7/ I leave the Bucket Menu.
8/ --> Bucket is totally denied.

I am going to check if the second IP I have informed in step 6 has been taken into account. That would result in a security breach.

[pauljegouic]

After some investigations thanks to @kevinisninja from GCAT, we have those conclusions:

• Access Policies are still editable after a IP firewall apply. But there's a trick. (see the GIF)

--> This leads me to the initial situation where COS has 2 endpoints, and It would be great that even if the Bucket content should not be access by an IP, it can still be managed through Terraform without getting a 403 error :)

If the console can do it, I bet Terraform teams can do it better than the GUI Team ;)

For now:

• AccessPolicies seems to be possible
• Configuration is not.

[kevinisninja]

Hey @hkantare, I think this might need to be pushed internally, and might be outside the scope of the provider.

The customer wants terraform to be able to refresh the state of the bucket properly, without whitelisting management infrastructure location IP

AFAIK This separation of control plane and data plane is possible today in ICD databases, where adding an IP whitelist will prevent access to the databases in the instance, but terraform is still able to configure/refresh the state of the instance

[kevinisninja]

@hkantare Do you know what exactly what attributes is the terraform provider trying to get that obtains a 403 failure?

Because when the bucket has an IP whitelist, when you go view it via the UI, you can still see the COS instance, as well as all the buckets that are in there. What information is the provider trying to obtain that causes it to fail, since it seems like you have all the information necessary in the UI.

Even with the IP firewall applied, you can still see the buckets that exist, so it doesn't seem like the provider should be failing, even if the system IP is not IP whitelisted.

[hkantare]

@kevinisninja terraform provider will try to set some of the bucket information like location, crn,..so on

terraform-provider-ibm/ibm/resource_ibm_cos_bucket.go

Line 121 in 0657d8c

func resourceIBMCOSRead(d *schema.ResourceData, meta interface{}) error {

Here you can find the set of code which we call to get the bucket information

[pauljegouic]

@hkantare

I've deep dive into the network call done on the GUI Console of IBM.

It seems that some information that you're mentionning are available (CRN, Location) even if my IP is not whitelisted.

My COS with 2 buckets, the first one apply a IP whitelist (and I am not doing those stuff through my IP.

Here is the call made from the GUI Console. Maybe something to investigate here with COS Team.

:authority: cloud.ibm.com
:method: POST
:path: /objectstorage/api/v1/crn%3Av1%3Abluemix%3Apublic%3Acloud-object-storage%3Aglobal%3Aa%2F67a4395fde4e4e0ca52f5554b249ceb7%3A1ded35ea-11df-47ab-9d0f-e28ebaacaf35%3A%3A/buckets?tzOffset=-120&bss_account=67a4395fde4e4e0ca52f5554b249ceb7
:scheme: https
accept: application/json, text/plain, */*
accept-encoding: gzip, deflate, br
accept-language: en,fr;q=0.9,en-US;q=0.8
content-length: 12
content-type: application/json;charset=UTF-8
cookie: userContext=0|fr|1; notice_preferences=2:; optimizelyEndUserId=oeu1590481436552r0.32841533613766494; BMAID=df4c11b9-7521-4754-8abf-62040f866cd5; CoreID6=730582 ...
[...]
credentials: same-origin
csrftoken: xXXXXWjQkZZtk_M
origin: https://cloud.ibm.com
referer: https://cloud.ibm.com/objectstorage/crn%3Av1%3Abluemix%3Apublic%3Acloud-object-storage%3Aglobal%3Aa%2F67a4395fde4e4e0ca52f5554b249ceb7%3A1ded35ea-11df-47ab-9d0f-e28ebaacaf35%3A%3A?paneId=manage
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36

Here is the response object:

{
   "containers":[
      {
         "name":"debug-gcat-bucket1",
         "creation_time":"06/05/2020 8:39:12 PM",
         "is_legacy":false,
         "has_activity_tracking":true,
         "storage_location":"eu-de-standard",
         "object_count":0,
         "bytes_used":0,
         "sse_kp_enabled":false,
         "bucket_lifecycle_policy":"Not Present",
         "bucket_protection_state":"DISABLED",
         "notifications_configuration":"Not Present",
         "metrics_monitoring_enabled":false,
         "location":"eu-de",
         "storage_class":"standard",
         "deleteAuthZ":{
            "action":"cloud-object-storage.bucket.delete_bucket",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35:bucket:debug-gcat-bucket1"
            },
            "subject":{
               "userId":"IBMid-5500030TRE"
            },
            "allowed":true
         },
         "viewKeyAuthZ":{
            "action":"cloud-object-storage.bucket.list_crk_id",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35:bucket:debug-gcat-bucket1"
            },
            "subject":{
               "userId":"IBMid-5500030TRE"
            },
            "allowed":true
         },
         "publicAccessAuthZ":{
            "action":"cloud-object-storage.object.get",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35:bucket:debug-gcat-bucket1"
            },
            "subject":{
               "userId":"IBMid-NoAuth"
            },
            "allowed":false
         },
         "putPolicyAuthZ":{
            "action":"iam.policy.create",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35::"
            },
            "subject":{
               "userId":"IBMid-5500030TRE"
            },
            "allowed":true
         },
         "isKPEnabled":false,
         "isCloudFunctionsEnabled":"disabled",
         "key":{

         }
      },
      {
         "name":"debug-gcat-buckettwo2",
         "creation_time":"06/05/2020 8:39:12 PM",
         "is_legacy":false,
         "has_activity_tracking":true,
         "storage_location":"eu-de-standard",
         "object_count":0,
         "bytes_used":0,
         "sse_kp_enabled":false,
         "bucket_lifecycle_policy":"Not Present",
         "bucket_protection_state":"DISABLED",
         "notifications_configuration":"Not Present",
         "metrics_monitoring_enabled":false,
         "location":"eu-de",
         "storage_class":"standard",
         "deleteAuthZ":{
            "action":"cloud-object-storage.bucket.delete_bucket",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35:bucket:debug-gcat-buckettwo2"
            },
            "subject":{
               "userId":"IBMid-XXXXX"
            },
            "allowed":true
         },
         "viewKeyAuthZ":{
            "action":"cloud-object-storage.bucket.list_crk_id",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35:bucket:debug-gcat-buckettwo2"
            },
            "subject":{
               "userId":"IBMid-XXXXX"
            },
            "allowed":true
         },
         "publicAccessAuthZ":{
            "action":"cloud-object-storage.object.get",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35:bucket:debug-gcat-buckettwo2"
            },
            "subject":{
               "userId":"IBMid-NoAuth"
            },
            "allowed":false
         },
         "putPolicyAuthZ":{
            "action":"iam.policy.create",
            "resource":{
               "crn":"crn:v1:bluemix:public:cloud-object-storage::a/67a4395fde4e4e0ca52f5554b249ceb7:1ded35ea-11df-47ab-9d0f-e28ebaacaf35::"
            },
            "subject":{
               "userId":"IBMid-XXXXX"
            },
            "allowed":true
         },
         "isKPEnabled":false,
         "isCloudFunctionsEnabled":"disabled",
         "key":{

         }
      }
   ],
   "is_truncated":false,
   "limit":"20"
}

[kevinisninja]

@hkantare per the information @ifs-pauljegouic provided,

the GUI is still able to get information regarding the buckets, even though he cannot access one of the buckets because his system ip is not whitelisted in the ip firewall.

so is there some other API with the COS team that they are using in the GUI that allows them to get that information, that maybe the provider should be using instead?

in the request paul posted, he is making a call to /objectstorage/api/v1/$CRN/buckets?tzOffset=-120&bss_account=accountid

do you know if that is the same API that schematics is calling to grab the information?
all i can see in the code is
apiEndpoint, apiEndpointPrivate := selectCosApi(parseBucketId(d.Id(), "apiType"), parseBucketId(d.Id(), "bLocation"))

[hkantare]

@kevinisninja The provider is using the https://github.com/IBM/ibm-cos-sdk-go/ below SDK to make calls to the API

[hkantare]

@kevinisninja This is the call failing when we add a firewall IP with 403 error code

terraform-provider-ibm/ibm/resource_ibm_cos_bucket.go

Line 166 in 0657d8c

bucketLocationConstraint, err := s3Client.GetBucketLocation(bucketLocationInput)

[kevinisninja]

@hkantare I talked with the COS team. They said that instead of using GetBucketLocation, you should be able to use ListBucketsExtendedOutput instead, and it will be able to provide you that information despite the IP firewall.
https://ibm.github.io/ibm-cos-sdk-go/service/s3/ListBucketsExtendedOutput.html

From their perspective, the bucket list is supposed to be visible, only Object and Bucket operations should be blocked.

Is it possible to refactor the provider code so that there will no longer be a 403 error?

[hkantare]

We changed to use ListBucketsExtendedOutput as per above comments.We are no longer getting 403 error now

The fix is available in master will be part of next release(25th june)

[pauljegouic]

Great !

[hkantare]

Fixed in latest release https://github.com/IBM-Cloud/terraform-provider-ibm/releases/tag/v1.8.0
https://github.com/IBM-Cloud/terraform-provider-ibm/blob/master/website/docs/r/cos_bucket.html.markdown

Cloud docs Docs will be updated shortly

[hkantare]

Available in latest releases

[pauljegouic]

Does not work properly

[hkantare]

Can you explain the scenario or the steps in which its not working properly

[pauljegouic]

So here is the point @hkantare

• ** 1 ** :this issue is about requesting the ability to CRUD Bucket configuration (ActivityTracker & SysDig), even if the IP is not allowed in the access policies.
• ** 2 ** : this issue is about requesting the ability to CRUD Access Policies, even if the IP is not allowed in the access policies.

Before, it was possible from the GUI to perform ** 2 **. Now it is not.
Before, it was possible from the TF to perform ** 2 **. Now it is possible.

Regarding ** 1 ** you have provided the ability to CREATE and READ Bucket Configuration, even if the IP is not allowed. UPDATE and DELETE is not allowed from a non-whitelisting IP.
Regarding ** 1 ** GUI does not provide this ability, even if the IP is not allowed.

To be honest my engineer are completly lost, because we cannot understand what's the actual capability of COS.

And there is the COS team quote:

"Anyone with Manager access, even if not logged in from a whitelisted IP address, can add or remove IPs from the Access Policies->Authorized IPs tab, but they will not have access to anything else, including bucket configuration, if they are not logged in from a whitelisted IP address. "

We have tested to provision a bucket with authorized IPs.

• I tried to refresh resource information from a non-authorized IPs, and Terraform says there is no changes (resource in input == existing resource)
• We have tried to change the sysdig or AT instance: Terraform see the changes, but won't apply it. (This is ok regarding the COS team answer)

This is leading me to the fact, that the AT and Sysdig information are not refreshed at all.

[umarali-nagoor]

Hi Pauljegouic,

Executed following scenario to test the “COS bucket activity tracker and ip filtering configuration” CRUD functionalities and its observed that these operations are working as expected.

Here are the steps and configurations that we have executed

Steps Followed:

1. Created VM Instance (say virtual machine with public ip 169.60.175.87 )

2. In local (mac) machine, include public ip of VM instance ( created in step-1 ) as part of allowed_ip list and create a COS bucket instance along with activity tracker

   allowed_ip =  [“169.60.175.87”]
   

3. Since VM instance public ip is part of white-list ip list, we should be able to perform update or delete operations from ip
   169.60.175.87

4. To do so, copied the terraform tf file and state file ( which was used to create the COS bucket in local machine ) from my local machine to VM instance

5. In Virtual machine, update activity tracker and allowed_ip configurations and run “terraform apply”

6. Able to update the same and verified from GUI side as well

7. Now try to destroy the COS bucket in VM instance by running “terraform destroy”, even the step is going through good

Configuration tf file

resource "ibm_resource_instance" "cos_instance" {
name = "cos-instance"
resource_group_id = data.ibm_resource_group.cos_group.id
service = "cloud-object-storage"
plan = "standard"
location = "global"
}

resource "ibm_resource_instance" "activity_tracker" {
name = "activity_tracker"
resource_group_id = data.ibm_resource_group.cos_group.id
service = "logdnaat"
plan = "lite"
location = "us-south"
}

resource "ibm_resource_instance" "metrics_monitor" {
name = "metrics_monitor"
resource_group_id = data.ibm_resource_group.cos_group.id
service = "sysdig-monitor"
plan = "lite"
location = "us-south"
}

resource "ibm_cos_bucket" "standard-ams03" {
bucket_name = "my1bucket-mine012090"
resource_instance_id = ibm_resource_instance.cos_instance.id
cross_region_location = "us"
storage_class = "standard"

activity_tracking {
read_data_events = true
write_data_events = true
activity_tracker_crn = ibm_resource_instance.activity_tracker.id
}

metrics_monitoring {
usage_metrics_enabled = true
metrics_monitoring_crn = ibm_resource_instance.metrics_monitor.id
}

allowed_ip = [“169.60.175.87”]

}

Can you please share the steps that you executed. Thanks in advance.

[pauljegouic]

In fact, here is the need:

• We are managing, administrating COS resource from our CICD pipeline,
• We are accessing the content of the bucket through the VPC (and we want to filter inbound with CSE VPC IPs)

Right now, I understand, that it is not possible. I should have had a meeting this afternoon with COS Team, but it was cancelled. I agree that the asked capability cannot be implemented by the TF Team only.

• followings.

Try to import an existing bucket

Try to redo the same process:

terraform init
terraform plan 
terraform apply

You will noticed that the plan does not see any changes.

As my team is used to Terraform, this mean, that the configuration retrieved by TF Provider, is the same as the one informed in the input. So we thought that the behaviour expected was implemented.

But in fact, it seems that the initial value is stored in the state, but is not refreshed with the actual configuration.

Try to import an existing bucket

Now perform terraform state rm resource.ibm_cos_bucket.standard-ams03 and terraform import resource.ibm_cos_bucket.standard-ams03 EXISTING_CRN

From your local machine (which acts with configuration as our pipeline - not whitelisted), and it will fail. This prove that the sysdigand activity_tracker are not refreshed.

In conclusion, this issue has been partially solved. (I think that all the misunderstood comes from the fact that this issue thread actually contains 2 issues ...)
We will discuss with the COS team, so that they can provide a solution on how to differentiate administration endpoints and usage endpoints (We would like the same behaviour regarding data & control as for ICD in fact).

[hkantare]

We are now closing the issue for now may be once we have updates from COS team you can open a new issue if it requires any changes from Terraform end..