GetEnvTS - Download Environmental Time-Series Data

Kostas Mammas, Data Scientist
mail: [email protected]

Introduction

GetEnvTS is an R interface for downloading data from www.ecad.eu. Using the interface of the tool the user is able to download dynamically the daily records of a set of environmmental variables and save them locally on a relational database model.

All the parameterizations of the tool are made through the config.yml file. The package is integrated with a postgresql database. The following parameters can be adjusted:

# Default Parameters
default:
  operating_system: "WINDOWS"    # Operating system (LINUX, MACOS, WINDOWS)
  num_cores:      4              # Number of cores
  blended:        FALSE          # Blended or non blended series
  schema_name:    "tran_ler"     # Name of database schema
  drop_schema:    TRUE           # Drop old schema
  create_schema:  TRUE           # Create new schema
  download_data:  TRUE           # Download data
  create_empty_tables: TRUE      # Create empty meteorological tables

# Available Meteorological Indices
indices:
  DailyMaxTemp:         FALSE
  DailyMinTemp:         FALSE
  DailyMeanTemp:        FALSE      
  DailyPrecipAmount:    TRUE  
  DailyMeanSeaLVLPress: FALSE
  DailyCloudCover:      FALSE
  DailyHumid:           TRUE
  DailySnowDepth:       FALSE
  DailySunShineDur:     FALSE
  DailyMeanWindSpeed:   FALSE
  DailyMaxWindGust:     FALSE
  DailyWindDirection:   TRUE

# Required packages; if not installed already then automatic installation will be performed
packages: ["data.table","RPostgreSQL", "stringr",
           "reshape2","yaml","markdown","rmarkdown", "gsubfn",
           "SqlRender", "htmltools","foreach", "doParallel", "DBI", "RODBC","odbc"
          ]

# Database configuaratopns
database_config:
  type:          "postgresql"
  username:      "username"
  password:      "password"
  database_name: "meteo_data"
  localhost:     "localhost"
  from_sql:      "postgresql"
  to_sql:        "postgresql"
  port:          5432

Running GetEnv

In command line, set the current path to the path of the code folder and run Rscript main.R: