data.txt

This file describes steps necessary to load data

============

Data is available from http://mta.info/developers/turnstile.html

============

# 1. Loading Remote-Booth-Station.xls

This data file has been converted to csv and saved as data/Remote-Booth-Station.csv

    # upload locally
    appcfg.py upload_data --config_file=bulkloader.yaml --filename=data/Remote-Booth-Station.csv --kind=Unit --url=http://127.0.0.1:8086/remote_api --application=mta-turnstiles --has_header

    # uploading remotely
    appcfg.py upload_data --config_file=bulkloader.yaml --filename=data/Remote-Booth-Station.csv --kind=Unit --application=mta-turnstiles --has_header .

# 2. parse into daily data

* Download the weekly txt file
* add to the summary csv file
$ cd scripts; for datafile in ../data/turnstile_10*.txt; do python turnstile_to_csv.py --input_file=$datafile; done
$ python turnstile_to_csv.py --input_file=turnstile_20100505.txt
$ python complete_dump_to_daily_files.py

# 3. load daily data

# upload locally
appcfg.py upload_data --config_file=bulkloader.yaml --filename=data/turnstile_daily.test.csv --kind=StationCount --url=http://127.0.0.1:8086/remote_api --application=mta-turnstiles  --batch_size=50

# uploading remotely
appcfg.py upload_data --config_file=bulkloader.yaml --filename=data/turnstile_daily.csv --kind=StationCount --application=mta-turnstiles --batch_size=50 .

# * appcfg.py upload_data --config_file=bulkloader.yaml --filename=data/turnstile_20100505.csv --kind=Unit .