Can use ETL modules as below by default.
| Step Class Name | Role |
|---|---|
| AzureBlobDownload | Download files from Azure Blob Storage |
| BigQueryRead | Read from bigquery table |
| FirestoreDocumentDownload | Download a document from Firestore |
| FtpDownload | Download a file via ftp |
| FtpDownloadFileDelete | Remove files downloaded via class 'FtpDownload' from FTP server. |
| GcsDownload | Download files from GCS |
| HttpDownload | Download a file via http |
| HttpDownloadViaBasicAuth | Download a file via HTTP with basic auth |
| MysqlRead | Execute a query to MySql server and get result as csv file |
| SftpDelete | Delete a file via SFTP |
| SftpDownload | Download a file via sftp |
| SftpDownloadFileDelete | Remove files downloaded via class 'SftpDownload' from SFTP server |
| S3Download | Download files from S3 |
| SqliteExport | Export a table data to csv |
| FirestoreDocumentDownload | Download a document from Firestore |
| Step Class Name | Role |
|---|---|
| ColumnLengthAdjust | Adjust columns of a csv file or a tsv file to the specified length. |
| CsvColumnExtract | Extract specific columns from csv files. |
| CsvConvert | Create new csv(tsv) file with given parameters |
| CsvMerge | Merge two csv files to a csv file |
| CsvConcat | Concat csv files |
| DateFormatConvert | Convert date format of columns of a csv file to another date format |
| ExcelConvert | Convert a excel file to a csv file |
| FileCompress | Compress a file |
| FileConvert | Convert file encoding |
| FileDecompress | Decompress a file |
| FileDivide | Divide a file to plural files |
| FileRename | Change file names with adding either prefix or suffix |
| ExecuteShellScript | Execute Shell Script |
| Step Class Name | Role |
|---|---|
| AzureBlobUpload | Upload files to Azure Blob Storage |
| BigQueryWrite | Read content from a file and insert it into a table of bigquery |
| FirestoreDocumentCreate | Create document |
| GcsUpload | Upload files to GCS |
| SftpUpload | Upload a file via sftp |
| SqliteImport | Read content from csv files and insert them into sqlite table |
| S3Upload | Upload files to S3 |
| GcsCopy | Upload files to GCS |
| Step Class Name | Role |
|---|---|
| SqliteQueryExecute | Execute query against sqlite tablecontent from |