Merged kfsone/tradedangerous into master

.gitignore

@@ -9,6 +9,8 @@ data/TradeDangerous.db
 data/TradeDangerous.db-journal
 data/TradeDangerous.prices
 data/*.stamp
+data/*.ini
+data/extra-stars.txt
 *.prices
 *.suo
 *.pyperf

CHANGES.txt

@@ -2,7 +2,51 @@
 TradeDangerous, Copyright (C) Oliver "kfsone" Smith, July 2014
 ==============================================================================
 
-[work in progress]
+v6.8.1 Jan 25 2015
+. (kfsone) Additional rules to prevent data pollution (ocr derp protection),
+. (kfsone) Support for maddavo's "3 hour" prices file,
+. (kfsone) Changed how downloads work to massively improve download speeds,
+. (kfsone) Improvements to the EDSC tools,
+. (kfsone) "First time" banner explaining the maddavo import plugin,
+. (kfsone) Fixed "shipvendor" command doing a poor job of finding stations,
+. (kfsone) "submit-distances" will now prompt you to correct distances that
+        EDSC reports as needing confirmation,
+. (kfsone) Fixed a crash when data was > 1 month old,
+. (kfsone) (API) Added a helper for clipboarding data,
+. (kfsone) (API) Added misc.edsc.StarSubmissionResult,
+. (kfsone) (API) Cleaned up various pieces of code documentation,
++ Data: Systems, Stations, Ships, lots of corrections/deletions too,
+
+v6.8.0 Jan 19 2015
+. (Dirk Wilhelm/kfsone) Added 'shipvendor' command,
+. (kfsone) Issue #135 show data age in checklist / on mfd
+. (kfsone) Issue #140 explain what/why 'requests' is required,
+. (kfsone) Issue #142 "--stations" with "nav" command caused error,
+. (kfsone) Added code to fight OCR Oerp,
+. (kfsone) '--ls-max' will now also unknown (0) distances,
+. (kfsone) Added '--max-routes' for setting an absolute max on how many
+        of the top routes we use after the first hop,
+. (kfsone) Added 'submit-distances' tool for submitting EDStarCoordinator
+        star data.
+. (kfsone) Added '--prune-score' and '--prune-hops' to run; these let
+        you discard routes that are under-performing early on which
+        can make calculating longer runs more efficient.
+. (kfsone) Added "--progress" to "run" to show the current hop,
+. (kfsone) "run" and TradeCalc are much smarter about which hops
+        they will consider when using --max-age, --blackmarket, etc.
+        (big perf win)
+. (kfsone) Renamed "misc/edstarquery.py" to "misc/edsc.py"
+        - Added misc.edsc.StarSubmission
+        - Renamed misc.edsc.EDStarQuery to misc.edsc.StarQuery
+        - Changed "submit-distances" to use "StarSubmission"
+
+v6.7.0 Jan 17 2015
+. (kfsone) ".prices" import automatically creates local placeholders
+        for unknown stations when using "-i", e.g:
+            trade.py buildcache -f -i
+            trade.py import --plug=maddavo  (this plugin sets -i for you)
+. (kfsone) Added a "--ls-max" option to "run" for filtering stations
+. (kfsone) +140 Systems
 . (tKE)  "buy" sub-command now supports ship names (find a ship vendor)
 . (kfsone) Partial code/documentation cleanup
 . (kfsone) Added a "getRoute" function to TradeDB()

README.txt

@@ -203,6 +203,51 @@ RUN sub-command:
   This command provides the primary trade run calculator functionality (it provides
   the functionality of the older TradeDangerous versions prior to 3.1)
 
+   Ship/Trade options:
+     --capacity N
+     --cap N
+       Maximum items you can carry on each hop.
+
+     --credits N
+     --cr N
+       How many credits to start with
+       e.g.
+         --credits 20000
+
+     --ly-per N.NN
+     --ly N.NN
+       Maximum distance your ship can jump between systems at full capacity.
+       NOTE: You can increase your range by selling your weapons.
+       e.g.
+         --ly-per 19.1
+         --ly-per 3
+
+     --empty-ly N.NN
+     --emp N.NN
+       DEFAULT: same as --ly-per
+       How far your ship can jump when empty (used by --start-jumps)
+
+     --limit N   DEFAULT: 0
+       If set, limits the maximum number of units of any cargo
+       item you will buy on any trade hop, incase you want to
+       hedge your bets or be a good economy citizen.
+       e.g.
+         --capacity 16 --limit 8
+
+     --insurance N   DEFAULT: 0
+     --ins N
+       How many credits to hold back for insurance purposes
+       e.g.
+         --insurance 1000
+         --ins 5000
+
+     --margin N.NN   DEFAULT: 0.01
+       At the end of each hop, reduce the profit by this much (0.02 = 2%),
+       to allow a margin of error in the accuracy of prices.
+       e.g.
+         --margin 0      (no margin)
+         --margin 0.01   (1% margin)
+
    Route options:
      --from <station or system>
        Lets you specify the starting station
@@ -238,6 +283,21 @@ RUN sub-command:
          --via Enterprise
          --via Chango
 
+     --hops N
+       DEFAULT: 2
+       Maximum number of hops (number of cargo pickups)
+       e.g.
+         --hops 8
+
+     --jumps-per N
+     --jum N
+       DEFAULT: 2
+       Limit the number of systems jumped to between each station
+       e.g.
+         -jumps-per 5
+
+
+   Filter options:
      --max-days-old N.NN
      -MD N.NN
        Filters out price data that exceeds a specified age in days
@@ -269,22 +329,31 @@ RUN sub-command:
          --ls-penalty 2.5
          --lsp=0              (disables this feature)
 
-     --unique
-     --uni
-       Only show routes which do not visit any station twice
-
-     --hops N
-       DEFAULT: 2
-       Maximum number of hops (number of cargo pickups)
+     --ls-max N
+       DEFAULT: 0
+       Filter stations by their distance-to-star. Stations for which
+       distance-to-star is known that have a distance above this will
+       not be considered for trading.
        e.g.
-         --hops 8
+         --ls-max 10000
+         --ls-m 32000
+
+     --prune-score N.NN
+       DEFAULT: 0
+       After a number of hops (controlled by --prune-hops), eliminate
+       candidate routes which have under-performed the leading candidate.
+       NOTE: This can speed up long run calculations, but it can also
+       cause you to miss gold-mines that are a just a few hops away...
+       e.g.
+         --prune-score 12.5   (prune routes scoring less than 10% of the leader)
 
-     --jumps-per N
-     --jum N
-       DEFAULT: 2
-       Limit the number of systems jumped to between each station
+     --prune-hops N
+       DEFAULT: 3
+       Being applying "--prune-score" from this hop onward. Set 0 to disable.
+       NOTE: This can speed up long run calculations, but it can also
+       cause you to miss gold-mines that are a just a few hops away...
        e.g.
-         -jumps-per 5
+         --prune-hop 4 --prune-score 22.5
 
      --avoid ITEM/SYSTEM/STATION
      --avoid AVOID,AVOID,...,AVOID
@@ -298,50 +367,9 @@ RUN sub-command:
          --avoid prise
          --av gold,aulin,enterprise,anderson
 
-   Ship/Trade options:
-     --capacity N
-     --cap N
-       Maximum items you can carry on each hop.
-
-     --credits N
-     --cr N
-       How many credits to start with
-       e.g.
-         --credits 20000
-
-     --ly-per N.NN
-     --ly N.NN
-       Maximum distance your ship can jump between systems at full capacity.
-       NOTE: You can increase your range by selling your weapons.
-       e.g.
-         --ly-per 19.1
-         --ly-per 3
-
-     --empty-ly N.NN
-     --emp N.NN
-       DEFAULT: same as --ly-per
-       How far your ship can jump when empty (used by --start-jumps)
-
-     --limit N   DEFAULT: 0
-       If set, limits the maximum number of units of any cargo
-       item you will buy on any trade hop, incase you want to
-       hedge your bets or be a good economy citizen.
-       e.g.
-         --capacity 16 --limit 8
-
-     --insurance N   DEFAULT: 0
-     --ins N
-       How many credits to hold back for insurance purposes
-       e.g.
-         --insurance 1000
-         --ins 5000
-
-     --margin N.NN   DEFAULT: 0.01
-       At the end of each hop, reduce the profit by this much (0.02 = 2%),
-       to allow a margin of error in the accuracy of prices.
-       e.g.
-         --margin 0      (no margin)
-         --margin 0.01   (1% margin)
+     --unique
+     --uni
+       Only show routes which do not visit any station twice
 
    Other options:
      --routes N   DEFAULT: 1
@@ -474,6 +502,12 @@ IMPORT sub-command:
       Any systems, stations, categories or items that aren't recognized
       by this version of TD will be reported but import will continue.
 
+      Unrecognized stations in the ".prices" file, or an import, will
+      have a placeholder station entry automatically created for them.
+
+      Note: When the cache is rebuilt, these stations will be lost, so
+      you may need to add the "-i" flag to the buildcache command.
+
 
 RARES sub-command: