[COST-5662] Add OpenShift virtual machine storage support

[djnakabaale]

Jira Ticket

COST-5662

Description

This change will:

• Add storage columns to the reporting_ocp_vm_summary_p table (COST-5911)
• Add SQL updates to populate storage columns in reporting_ocp_vm_summary_p (COST-5913)
• Update virtual machine endpoint response to include storage info (COST-5930)

Testing

1. Checkout Branch

2. Restart Koku

3. load test data

4. Hit endpoint

   http://127.0.0.1:8000/api/cost-management/v1/reports/openshift/resources/virtual-machines
   

   • You should see storage information in the response similar to this

   "data": [
       {
           "date": "2025-01",
           "vm_names": [
               {
                   "vm_name": "test",
                   "values": [
                       {
                           "vm_name": "test",
                           "date": "2025-01",
                           "cluster": "Test OCP on Premises",
                           "node": "compute_1",
                           "project": "Raleigh",
                           "source_uuid": [
                               "04ebdf16-63fc-46a1-abf6-377d94110bea"
                           ],
                           "storage": [
                               {
                                   "usage": 0.417231182795699,
                                   "capacity": 0.645161290322581,
                                   "pvc_name": "raleigh_pod_name2_data",
                                   "storage_class": "hostpath",
                                   "units": "GiB-Mo"
                               }
                           ],
                           "tags": [
                               {
                                   "key": "application",
                                   "values": [
                                       "Phoenix"
                                   ]
                               },
                               {
                                   "key": "instance-type",
                                   "values": [
                                       "medium"
                                   ]
                               },
                               {
                                   "key": "vm_kubevirt_io_name",
                                   "values": [
                                       "test"
                                   ]
                               }
                           ],
                           // other fields
                       }
                   ]
               }
           ]
       }
   ]
   

5. Try exporting to a CSV

   curl --location --globoff 'http://127.0.0.1:8000/api/cost-management/v1/reports/openshift/resources/virtual-machines' --header 'Accept: text/csv' >> output.csv
   

   • You should see all array fields (like source_uuid, storage and tags) flattened, with headers similar to this:

   source_uuid.0	storage.0.capacity	storage.0.pvc_name	storage.0.storage_class	storage.0.units	storage.0.usage	storage.1.capacity	storage.1.pvc_name	storage.1.storage_class	storage.1.units	storage.1.usage	tags.0.key	tags.0.values.0	tags.1.key	tags.1.values.0	tags.2.key	tags.2.values.0
   
   

Release Notes

• proposed release note

* [COST-5662](https://issues.redhat.com/browse/COST-5662) Add OpenShift virtual machine storage support

[codecov]

Codecov Report

Attention: Patch coverage is 94.87179% with 2 lines in your changes missing coverage. Please review.

Project coverage is 94.1%. Comparing base (7b2b6dd) to head (7663a97).

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #5515   +/-   ##
=====================================
  Coverage   94.1%   94.1%           
=====================================
  Files        371     371           
  Lines      31574   31584   +10     
  Branches    3385    3385           
=====================================
+ Hits       29716   29732   +16     
+ Misses      1206    1199    -7     
- Partials     652     653    +1     

[myersCody]

https://ci.ext.devshift.net/job/koku-pipeline-pr-check-main/4747/

• 3 failures
  2 are time out failures
  1 is a known issue

[myersCody]

/retest

[myersCody]

/retest

[lcouzens]

The last 2 runs had the following tests fail, test_api_ocp_ingest_source_ros_data and test_api_ocp_ros_report_upload both of which were ingest time outs. Highly likely to be unrelated to these changes. Perhaps @project-koku/quality-engineers can sign off on this one to not have to wait for another run?

[myersCody]

Successful smokes: https://ci.ext.devshift.net/job/koku-pipeline-pr-check-main/4767/

[djnakabaale]

/retest

[myersCody]

/retest