Minor edits

Author: ozsolarwind

SIREN_notes.html

@@ -33,10 +33,11 @@ <h2 id="windows">Windows package</h2>
 <li>If required, the sample SIREN simulation of the SWIS - <em>siren_sample.exe</em>
 </ul>
 <h3>Packaging for Windows</h3>
-<p>I use PyInstaller (currently Version 5.6.2) to package SIREN for Windows. I use the multi-package facility to create executables for <code>siren.py, sirenm.py</code> and the tools (<code>flexiplot.py, getmap.py, getmerra2.py, indexweather.py, makegrid.py, makeweatherfiles.py, powermatch.py, powerplot.py, sirenupd.py</code> and <code>updateswis.py</code>). <code>siren_application.spec</code> is the spec file (<code>pyinstaller.exe siren_application.spec</code> with options <code>--log-level ERROR</code> and/or <code>--noconfirm</code> ... <code>pyinstaller.exe --noconfirm --log-level ERROR siren_application.spec</code>). This enables all programs to be included in a single library for distribution. PyInstaller places them in individual directories - I use <code>package_siren.py</code> to copy all the executables into the <em>siren</em> directory and remove some unwanted files that are included as part of the packaging process.</p>
+<p>I use PyInstaller (currently Version 5.6.2) to package SIREN for Windows. I use the multi-package facility to create executables for <code>siren.py, sirenm.py</code> and the tools (<code>flexiplot.py, getera5.py, getmap.py, getmerra2.py, indexweather.py, makegrid.py, makeweatherfiles.py, pmtmldetail.py, powermatch.py, powerplot.py, sirenupd.py</code> and <code>updateswis.py</code>). <code>siren_application.spec</code> is the spec file (<code>pyinstaller.exe siren_application.spec</code> with options <code>--log-level ERROR</code> and/or <code>--noconfirm</code> ... <code>pyinstaller.exe --noconfirm --log-level ERROR siren_application.spec</code>). This enables all programs to be included in a single library for distribution. PyInstaller places them in individual directories - I use <code>package_siren.py</code> to copy all the executables into the <em>siren</em> directory and remove some unwanted files that are included as part of the packaging process.</p>
 <p>All these files have a version file where the version is <code>&lt;major&gt;,&lt;minor&gt;.&lt;yyyy&gt;.&lt;mmdd&gt;</code>. <code>&lt;major&gt;</code> and <code>&lt;minor&gt;</code> refer to the SIREN release. The date part is the date the program was last changed. The version numbering simulates Microsoft version numbers. <code>template_version</code> provides a template for the version files to enable new SIREN releases to be packaged. The current <code>major</code> version is <strong><code>4</code></strong> (<code>1</code> was the initial version, <code>2</code> for the change from Python2 to Python3, and for the change from PyQt4 to PyQt5, and <code>4</code> for a change to a new packaging environment, Windows 10 rather than Windows 7, and a change to 64-bit Python).</p>
 <p>To enable the modules to be built correctly most require <code>hiddenimports</code>, as identified in <code>siren_application.spec</code>, as often PyInstaller will fail to identify all imports from libraries used by the SIREN program.</p>
-<p><code>makeweather_getmerra2.spec</code> and <code>package_makeweatherfiles.py</code> package <code>makeweatherfiles.py</code> and <code>getmerra2.py</code> to provide a standalone tool to create SAM weather files from MERRA-2 datasets.</p>
+<h4>Makeweatherfiles</h4>
+<p>Weather files suitable for SAM can be obtained from either ERA5 or MERRA-2 satellite data. I again use the multi-package facility to create executables for <code>getera5.py, getmerra2.py</code> and <code>makeweatherfiles.py</code> to provide a standalone tool to create SAM weather files from ERA5 or MERRA-2 datasets. <code>package_makeweatherfiles.py</code> packages these three programs.</p>
 <h2 id="python">A Python version</h2>
 <p>A Python version of SIREN is a viable alternative for Linux and MacOS (and for Windows users who have Python installed). The following instructions may assist:</p>
 <ul>
@@ -66,7 +67,7 @@ <h2 id="github">github</h2>
 <li><em>siren_application.spec</em> is the specification file used to package the SIREN application using PyInstaller</li>
 <li>You can also view the SIREN help file at <a href="https://rawgit.com/ozsolarwind/siren/master/help.html" target="_blank">https://rawgit.com/ozsolarwind/siren/master/help.html</a>
 </ul>
-<p>You should download all files (except the spec file) to your target &ldquo;home&rdquo; folder for SIREN.<p>
+<p>You should download all files (except the package and spec files) to your target &ldquo;home&rdquo; folder for SIREN.<p>
 <h2>SourceForge</h2>
 <p><a href="https://sourceforge.net/projects/sensiren/" target="_blank">https://sourceforge.net/projects/sensiren/</a> contains a number of self-extracting zip files for the full SIREN Toolkit and a number of the components. See <a href="help.html#install" target="_blank">Installation</a>.
 <ul>
@@ -97,7 +98,7 @@ <h2 id="libraries">Python Libraries</h2>
 <tr><td>python3-openpyxl</td></tr>
 <tr><td>os</td><td style="text-align:center">y</td></tr>
 <tr><td>pwd</td><td style="text-align:center">y</td><td>Linux ?</td></td></tr>
-<tr><td><s>pyexcel</s></td><td>&nbsp;</td><td><s>Support for spreadsheets</s></td></td></tr>
+<tr><td><s>pyexcel</s></td><td>&nbsp;</td><td><s>Support for spreadsheet. No longer used</s></td></td></tr>
 <tr><td><s>pyexcel-io</s></td><td>&nbsp;</td><td><s>CSV files</s></td></td></tr>
 <tr><td><s>pyexcel-ods</s></td><td>&nbsp;</td><td><s>ODS (LibreOffice) files</s></td></td></tr>
 <tr><td><s>pyexcel-xls</s></td><td>&nbsp;</td><td><s>XLS (Excel) files</s></td></td></tr>
@@ -121,8 +122,7 @@ <h2 id="libraries">Python Libraries</h2>
 <tr><td>urllib3</td></tr>
 <tr><td>webbrowser</td><td style="text-align:center">y</td></tr>
 <tr><td>win32api</td><td>&nbsp;</td><td>Windows</td></tr>
-<tr><td>xlrd</td></td><td>&nbsp;</td><td>At present needs to be an earlier version. This can be done by setting the version on install e.g.<br />
-<code>pip install xlrd==1.2.0</code></td></tr>
+<tr><td>xlrd</td></tr>
 <tr><td>xlwt</td></tr>
 <tr><td>xml.etree.ElementTree</td><td style="text-align:center">y</td></tr>
 <tr><td>zipfile</td><td style="text-align:center">y</td></tr>
@@ -152,7 +152,9 @@ <h2 id="source">SIREN Programs</h2>
 <tr><td><del>makerainfall2.py</del></td><td><del>Create rainfall files from MERRA-2 data</del></td></tr>
 <tr><td>makeweatherfiles.py (*)</td><td>Standalone program to create weather files from MERRA-2 data</td></tr>
 <tr><td>newstation.py</td><td>Create/edit a new power station</td></tr>
+<tr><td>plot3d.py</td><td>A prototype and incomplete program to produce 3D plots (charts) for Powermatch batch</td></tr>
 <tr><td>plotweather.py (*)</td><td>Plot (chart) weather data</td></tr>
+<tr><td>pmtmldetail.py</td><td>Produce a more detailed Powermatch Detail worksheet which contains hourly details on how RE is used to meet load</td></tr>
 <tr><td>powerclasses.py</td><td>Classes used by PowerModel (powermodel.py)</td></tr>
 <tr><td>powermatch.py</td><td>Match generation to load. Produce costed scenarios</td></tr>
 <tr><td>powermodel.py (*)</td><td>Process SAM models for stations to produce spreadsheets and charts</td></tr>
@@ -193,5 +195,5 @@ <h3>sam_models_list</h3>
 </ul>
 <h3>opt_debug</h3>
 <p><code>opt_debug.py</code> can be used to debug the <code>powermatch</code> optimisation routine. It doesn't need to be packaged with SIREN.</p>
-<p>Angus King<br />November 2022</p>
+<p>Angus King<br />August 2023</p>
 </body>

help.html

@@ -3099,6 +3099,11 @@ <h3 id="mouse">Right Mouse Click</h3>
 </table>
 <p><a href="#help">Back to top</a></p>
 <h2 id="usage">Using the Simulation</h2>
+<p>The simulation has two parts:</p>
+<ol>
+<li>Using SIREN to build scenarios for an energy mix as described in more detail below</li>
+<li>Using the Powermatch data produced to complete a costed renewable energy scenario. This is described in <a href="#powermatch">Powermatch</a></li>
+</ol>
 <p>The simulation allows you to build scenarios for the energy mix for your area of interest. It is based around the map of your chosen area and allows you to visualise the layout of the existing electricity network plus the additional stations you place on the map to build your scenario(s). The map initially shows the main skeleton of the existing electricity grid infrastructure and current generation stations.</p>
 <p>To add a new station simply right-click on the map at the desired location and then update the details for the station (Name, Technology, Capacity and so forth) and press <em>Save &amp; Exit</em>. You can also copy an existing station and other options as specified above. </p>
 <p>As you build your scenario, the simulation adds additional infrastructure to connect the new generation plants to the grid. The colours for the majority of the items on the map are as you specify in the Preferences file. You navigate around the map using normal mouse movements to scroll, zoom, etc. and interact with it as described previously (<a href="#menus">Menus</a>). Using the various menu options you can model your whole scenario, or by right-clicking you can deal with an individual station. There are a number of additional windows, such as menu options, that can be opened to assist with using SIREN (see <a href="#windows">Windows</a>)</p>
@@ -4030,7 +4035,7 @@ <h2 id="feedback">Feedback</h2>
 <p><a href="#help">Back to top</a></p>
 <h2 id="files">SIREN File Formats</h2>
 <p>SIREN accepts a number of file formats for each of the data files it uses. In many cases input files can be either a CSV or Excel (XLS or XLSX) file. Due to the initial source for each data file the format for each may be different. For CSV and Excel files the first cell for each column is usually taken as the column name.</p>
-<p><strong>Note:</strong> Other than the Powermatch file, all Excel files output by SIREN are currently in XLS format (XLSX files require a different Python library to create them).</p>
+<p><strong>Note:</strong> Other than the Powermatch file, all Excel files output by SIREN are currently in XLS format (XLSX files require a different Python library to create them). Powermatch itself produces XLSX files.</p>
 <p>The files and formats are as follow:</p>
 <table border="0", class="none" style="margin-left:35px">
 <tr class="none">
@@ -4359,7 +4364,7 @@ <h5>getera5</h5>
 <br />
 <strong>Note:</strong> The behaviour of the bounding coordinates fields are a little quirky in that they will adjust themselves so the west (left) value is always west of the east value and the north (upper) value is always higher than the south value
 </li>
-<li><em>Grid</em>. The size of the desired grid latitude and longitude. These can be in increments of 0.25 degrees up to a mximum of 3 degrees</li>
+<li><em>Grid</em>. The size of the desired grid latitude and longitude. These can be in increments of 0.25 degrees up to a maximum of 3 degrees</li>
 <li><em>Start month</em>. The first month to be downloaded. The CDS API is flexible on the range of dates that can be requested. For pragmatic purposes <code>getera5</code> requests data as a range of months</li>
 <li><em>End month</em>. The last month to be downloaded. If a Start month of 1 and an End month of 12 is chosen the program will request a year of data as a single file</li>
 <li><em>Target Folder</em>. Use this to specify the location for the ERA5 files. To change the location you need to click on the field and navigate to the correct folder (you can't type into the field)</li>

package_siren.py

@@ -0,0 +1,44 @@
+#!/usr/bin/python3
+#
+import os
+import shutil
+import sys
+
+def delist_dir(directory):
+    for filename in os.listdir(directory):
+        if os.path.isdir(directory + filename):
+            delist_dir(directory + filename + '\\')
+      #      print directory + filename + '\\'
+            os.rmdir(directory + filename + '\\')
+        else:
+            if filename == 'epsg':
+                continue
+            os.remove(directory + filename)
+
+# \dist\siren will contain sirenm and most files
+dirs = ['app_flexiplot', 'app_getera5', 'app_getmap', 'app_getmerra2', 'app_indexweather',
+        'app_makegrid', 'app_makeweatherfiles', 'app_powermatch', 'app_powerplot',
+        'app_siren', 'app_sirenupd', 'app_updateswis']
+for di in dirs:
+    print('Including:', di)
+    fils = sorted(os.listdir('dist\\' + di))
+    for fil in fils:
+        if os.path.isdir('dist\\' + di + '\\' + fil):
+            print('Directory encountered:', fil)
+            try:
+                shutil.copytree('dist\\' + di + '\\' + fil, 'dist\\siren\\' + fil)
+            except:
+                pass
+        else:
+            try:
+                shutil.copy2('dist\\' + di + '\\' + fil, 'dist\\siren')
+            except:
+                pass
+
+del_dirs = [] #'tcl\\tzdata', 'tcl\\encoding', 'mpl-data\\fonts', 'pytz\\zoneinfo',
+           # 'mpl_toolkits\\basemap\\data']
+for di in del_dirs:
+    if os.path.exists('dist\\siren\\' + di):
+        print('Removing:', di)
+        src_dir = 'dist\\siren\\' + di + '\\'
+        delist_dir(src_dir)

powerplot.py

@@ -82,7 +82,7 @@ def set_hatch(self, item):
     def replace_words(self, what, src, tgt):
         words = {'m': ['$MTH$', '$MONTH$'],
                  'y': ['$YEAR$'],
-                 's': ['$SHEET']}
+                 's': ['$SHEET$']}
         tgt_str = src
         if tgt == 'find':
             for wrd in words[what]: