diff --git a/README.md b/README.md index 1e7f8dba4..5a0618d2e 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Allsky Camera ![Release](https://img.shields.io/badge/Version-v2023.05.01-green.svg) [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=MEBU2KN75G2NG&source=url) +# Allsky Camera ![Release](https://img.shields.io/badge/Version-v2023.05.01_01-green.svg) [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=MEBU2KN75G2NG&source=url) This is the source code for the Allsky Camera project described [on Instructables](http://www.instructables.com/id/Wireless-All-Sky-Camera/).   @@ -6,25 +6,26 @@ This is the source code for the Allsky Camera project described [on Instructable

-> **This README and the [Allsky documentation](https://github.com/thomasjacquin/allsky/wiki) will help get your allsky camera up and running.** +> **This README and the [Allsky documentation](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/index.html) will help get your allsky camera up and running.**   ## Requirements -You will need the following hardware: +You will need the following: - * A camera (Raspberry Pi HQ, Module 3 or compatible, or ZWO ASI) - * A Raspberry Pi (2, 3, 4 or Zero 2 W). + * A Raspberry Pi (Zero 2, 2, 3, 4) running Pi OS. + * A camera (Raspberry Pi HQ, Module 3, or RPi compatible, or ZWO ASI)   > **NOTES:** -> - The ZWO ASI120-series cameras are not recommended due to somewhat poor quality. See the [Troubleshooting --> ZWO Cameras](https://github.com/thomasjacquin/allsky/wiki) documentation for notes on the ASI120-series and related T7 / T7C cameras. -> - The Pi Zero and Pi Zero W, with their limited memory and _very_ limited CPU power (single CPU core), are not recommended. You will likely not be able to create keograms, startrails, or timelapse videos. -> - The Pi Zero 2 and Pi Zero 2 W, with their limited memory and somewhat limited CPU power, are not recommended unless cost is a major concern. Creating keograms, startrails, and timelapse videos may or may not be possible. -> - We have not found any "Pi-compatible" boards that are actually compatible, so buyer beware. +> - Only the Raspberry Pi OS is supported. Other operating systems like Ubuntu are NOT supported. +> - The ZWO ASI120-series cameras are not recommended due to somewhat poor quality. See [Troubleshooting --> ZWO Cameras](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/troubleshooting/ZWOCameras.html) for notes on the ASI120-series and related T7 / T7C cameras. +> - The Pi Zero with its limited memory and _very_ limited CPU power (single CPU core), is not recommended. You will likely not be able to create keograms, startrails, or timelapse videos. +> - The Pi Zero 2 with its limited memory and somewhat limited CPU power, is not recommended unless cost is the only concern. Creating keograms, startrails, and timelapse videos may or may not be possible. +> - The Le Potato is the only "Pi-compatible" board that we've found to actually be compatible, so buyer beware. --- @@ -35,7 +36,7 @@ You will need the following hardware: PatriotAstro created a great [video](https://www.youtube.com/watch?v=7TGpGz5SeVI) describing the installation steps below. **We highly suggest viewing it before installing the software.** -Detailed installation instructions can be found in the [Installing / Upgrading --> Allsky](https://github.com/thomasjacquin/allsky/wiki) documentation. +Detailed installation instructions can be found at [Installing / Upgrading --> Allsky](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/installations/Allsky.html). --- @@ -45,7 +46,7 @@ Detailed installation instructions can be found in the [Installing / Upgrading - ## Web User Interface (WebUI)

- +

The WebUI is now installed as part of Allsky and is used to administer Allsky, and to a lesser extent, your Pi. It can also be used to view the current image as well as all saved images, keograms, startrails, and timelapse videos. @@ -67,7 +68,7 @@ If it is behind a firewall consult the documentation for your network equipment By installling the optional Allsky Website you can display your files on a website on the Pi, on another machine, or on both. -See the [Installation / Upgrading --> Website](https://github.com/thomasjacquin/allsky/wiki) documentation for information on how to install and configure an Allsky Website. +See [Installation / Upgrading --> Website](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/installations/AllskyWebsite.html) for information on how to install and configure an Allsky Website. --- @@ -82,7 +83,7 @@ Allsky supports running "modules" after each picture is taken to change the imag The Overlay Editor lets you easily specify what text and images you want in your overlay, and place them using drag-and-drop. Each field can be formatted however you want (font, color, size, position, rotation, etc.). The only limit is your imagination!! -See the [Explanations / How To -> Overlays](https://github.com/thomasjacquin/allsky/wiki) and [Explanations / How To -> Modules](https://github.com/thomasjacquin/allsky/wiki) documentation for more information. +See [Explanations / How To -> Overlays](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/overlays/overlays.html) and [Explanations / How To -> Modules](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/modules/modules.html) for more information. --- @@ -93,7 +94,7 @@ See the [Explanations / How To -> Overlays](https://github.com/thomasjacquin/all Dark frame subtraction removes hot pixels from images by taking images at different temperatures with a cover on your camera lens and subtracting those images from nighttime images. -See the [Explanations / How To -> Dark frames](https://github.com/thomasjacquin/allsky/wiki) documentation for more information. +See [Explanations / How To -> Dark frames](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/explanations/darkFrames.html) for more information. --- @@ -120,7 +121,7 @@ By default, a timelapse video is generated at the end of nighttime from all of t A **Keogram** is an image giving a quick view of the day's activity. For each image a central vertical column 1 pixel wide is extracted. All these columns are then stitched together from left to right. This results in a timeline that reads from dawn to the end of nighttime (the image above only shows nighttime data since daytime images were turned off). -See the [Explanations / How To --> Keograms](https://github.com/thomasjacquin/allsky/wiki) documentation. +See [Explanations / How To --> Keograms](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/explanations/keograms.html). --- @@ -137,7 +138,7 @@ See the [Explanations / How To --> Keograms](https://github.com/thomasjacquin/al **Startrails** are generated by stacking all the images from a night on top of each other. In the image above, Polaris is centered about one-fourth the way from the top. -See the [Explanations / How To --> Startrails](https://github.com/thomasjacquin/allsky/wiki) documentation. +See [Explanations / How To --> Startrails](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/explanations/startrails.html). --- @@ -150,7 +151,7 @@ See the [Explanations / How To --> Startrails](https://github.com/thomasjacquin/ You can specify how many days worth of images to keep in order to keep the Raspberry Pi SD card from filling up. If you have the Allsky Website installed on your Pi, you can specify how many days worth of its imags to keep. -See the **DAYS_TO_KEEP** and **WEB_DAYS_TO_KEEP** settings in the [Settings --> Allsky Website](https://github.com/thomasjacquin/allsky/wiki) documentation. +See the **DAYS_TO_KEEP** and **WEB_DAYS_TO_KEEP** settings in [Settings --> Allsky](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/settings/allsky.html). --- @@ -161,7 +162,7 @@ See the **DAYS_TO_KEEP** and **WEB_DAYS_TO_KEEP** settings in the [Settings --> ## Share your sky -If you want your allsky camera added to the [Allsky map](http://www.thomasjacquin.com/allsky-map), see the [Put your camera on Allsky Map](https://github.com/thomasjacquin/allsky/wiki) documentation. +If you want your allsky camera added to the [Allsky map](http://www.thomasjacquin.com/allsky-map), see [Put your camera on Allsky Map](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/miscellaneous/AllskyMap.html). If you know anyone in Greenland or Antartica, send them a camera!! @@ -179,7 +180,7 @@ If you know anyone in Greenland or Antartica, send them a camera!! ## Release changes See the -[Allsky Version Change Log](https://github.com/thomasjacquin/allsky/blob/master/html/documentation/changeLog.html) +[Allsky Version Change Log](https://htmlpreview.github.io/?https://raw.githubusercontent.com/thomasjacquin/allsky/master/html/documentation/changeLog.html) for a list of changes in this release and all prior releases. --- diff --git a/allsky.sh b/allsky.sh index 9ed29bb16..a6bda8bdd 100755 --- a/allsky.sh +++ b/allsky.sh @@ -32,6 +32,27 @@ source "${ALLSKY_CONFIG}/config.sh" || exit ${ALLSKY_ERROR_STOP} #shellcheck disable=SC2086 source-path=scripts source "${ALLSKY_SCRIPTS}/installUpgradeFunctions.sh" || exit ${ALLSKY_ERROR_STOP} +# Make sure they rebooted if they were supposed to. +NEEDS_REBOOT="false" +reboot_needed && NEEDS_REBOOT="true" + +# Make sure the settings have been configured after an installation or upgrade. +LAST_CHANGED="$( settings ".lastChanged" )" +if [[ ${LAST_CHANGED} == "" || ${LAST_CHANGED} == "null" ]]; then + echo "*** ===== Allsky needs to be configured before it can be used. See the WebUI." + if [[ ${NEEDS_REBOOT} == "true" ]]; then + echo "*** ===== The Pi also needs to be rebooted." + doExit "${EXIT_ERROR_STOP}" "Error" \ + "Allsky needs\nconfiguration\nand the Pi needs\na reboot" \ + "Allsky needs to be configured then the Pi rebooted." + else + doExit "${EXIT_ERROR_STOP}" "ConfigurationNeeded" "" "" + "${ALLSKY_SCRIPTS}/addMessage.sh" "Error" "Allsky needs to be configured." + fi +elif [[ ${NEEDS_REBOOT} == "true" ]]; then + doExit "${EXIT_ERROR_STOP}" "RebootNeeded" "" "" + "${ALLSKY_SCRIPTS}/addMessage.sh" "Error" "The Pi needs to be rebooted." +fi SEE_LOG_MSG="See ${ALLSKY_LOG}" ARGS_FILE="${ALLSKY_TMP}/capture_args.txt" @@ -171,6 +192,12 @@ else "${NOT_STARTED_MSG}
${MSG}" fi +# Make sure the settings file is linked to the camera-specific file. +if ! MSG="$( check_settings_link "${SETTINGS_FILE}" )" ; then + "${ALLSKY_SCRIPTS}/addMessage.sh" "error" "${MSG}" + echo "ERROR: Settings file (${SETTINGS_FILE}) not linked correctly." >&2 +fi + # Make directories that need to exist. if [[ -d ${ALLSKY_TMP} ]]; then # remove any lingering old image files. @@ -186,11 +213,13 @@ else "${ALLSKY_SCRIPTS}/addMessage.sh" warning "${ME}: ${MSG}" fi +rm -f "${ALLSKY_BAD_IMAGE_COUNT}" # Start with no bad images + # Clear out these files and allow the web server to write to it. -: > "${ALLSKY_ABORTEDUPLOADS}" -: > "${ALLSKY_ABORTEDTIMELAPSE}" -sudo chgrp "${WEBSERVER_GROUP}" "${ALLSKY_ABORTEDUPLOADS}" "${ALLSKY_ABORTEDTIMELAPSE}" -sudo chmod 664 "${ALLSKY_ABORTEDUPLOADS}" "${ALLSKY_ABORTEDTIMELAPSE}" +rm -fr "${ALLSKY_ABORTS_DIR}" +mkdir "${ALLSKY_ABORTS_DIR}" +sudo chgrp "${WEBSERVER_GROUP}" "${ALLSKY_ABORTS_DIR}" +sudo chmod 775 "${ALLSKY_ABORTS_DIR}" # Optionally display a notification image. if [[ $USE_NOTIFICATION_IMAGES -eq 1 ]]; then @@ -219,17 +248,12 @@ if [[ -z ${LOCALE} || ${LOCALE} == "null" ]]; then fi fi -# shellcheck disable=SC2207 -KEYS=( $(settings 'keys[]') ) -for KEY in "${KEYS[@]}" -do - K="$(settings ".${KEY}")" - # We must pass "-config ${ARGS_FILE}" on the command line, - # and debuglevel we did above, so don't do them again. - [[ ${KEY} == "config" && ${KEY} == "debuglevel" ]] && continue - - echo "-${KEY}=${K}" >> "${ARGS_FILE}" -done +# We must pass "-config ${ARGS_FILE}" on the command line, +# and debuglevel we did above, so don't do them again. +TAB="$( echo -e "\t" )" +convert_json_to_tabs "${SETTINGS_FILE}" | + grep -E -i -v "^config${TAB}|^debuglevel${TAB}" | + sed -e 's/^/-/' -e "s/${TAB}/=/" >> "${ARGS_FILE}" # When using a desktop environment a preview of the capture can be displayed in a separate window. # The preview mode does not work if we are started as a service or if the debian distribution has no desktop environment. @@ -252,6 +276,9 @@ CAPTURE="capture_${CAMERA_TYPE}" rm -f "${ALLSKY_NOTIFICATION_LOG}" # clear out any notificatons from prior runs. +# Clear up any flow timings +"${ALLSKY_SCRIPTS}/flow-runner.py" --cleartimings + # Run the main program - this is the main attraction... # Pass debuglevel on command line so the capture program knows if it should display debug output. "${ALLSKY_BIN}/${CAPTURE}" -debuglevel "${ALLSKY_DEBUG_LEVEL}" -config "${ARGS_FILE}" diff --git a/assets/WebUI.png b/assets/WebUI.png deleted file mode 100644 index cb7d3b269..000000000 Binary files a/assets/WebUI.png and /dev/null differ diff --git a/config_repo/config.sh.repo b/config_repo/config.sh.repo index 867c31e2d..ac72d43da 100644 --- a/config_repo/config.sh.repo +++ b/config_repo/config.sh.repo @@ -156,7 +156,8 @@ UHUBCTL_PORT=2 # ================ DO NOT CHANGE ANYTHING BELOW THIS LINE ================ ME2="$(basename "${BASH_SOURCE[0]}")" -CAMERA_TYPE="" # Updated during installation +# CAMERA_TYPE is updated during installation +CAMERA_TYPE="" if [ "${CAMERA_TYPE}" = "" ]; then echo -e "${RED}${ME2}: ERROR: Please set 'Camera Type' in the WebUI.${NC}" sudo systemctl stop allsky > /dev/null 2>&1 @@ -192,6 +193,7 @@ CAMERA_MODEL="$(settings '.cameraModel')" # So scripts can conditionally output messages. ALLSKY_DEBUG_LEVEL="$(settings '.debuglevel')" -ALLSKY_VERSION="XX_ALLSKY_VERSION_XX" # Updated during installation +# ALLSKY_VERSION is updated during installation +ALLSKY_VERSION="XX_ALLSKY_VERSION_XX" CONFIG_SH_VERSION=1 diff --git a/config_repo/ftp-settings.sh.repo b/config_repo/ftp-settings.sh.repo index 46c2ccae0..89c91ebe3 100644 --- a/config_repo/ftp-settings.sh.repo +++ b/config_repo/ftp-settings.sh.repo @@ -41,7 +41,7 @@ REMOTE_HOST="" REMOTE_PORT="" -############### ftp, ftps, and sftp PROTOCOLS only: +############### ftp, ftps, and sftp PROTOCOLS only. REMOTE_USER is also used by the scp PROTOCOL: # The username of the login on the remote server. REMOTE_USER="" diff --git a/config_repo/lighttpd.conf.repo b/config_repo/lighttpd.conf.repo index 28d9626a5..754b5df15 100644 --- a/config_repo/lighttpd.conf.repo +++ b/config_repo/lighttpd.conf.repo @@ -20,6 +20,7 @@ setenv.add-environment = ( alias.url = ("/current/" => "XX_ALLSKY_HOME_XX/") alias.url += ("/images/" => "XX_ALLSKY_IMAGES_XX/") +alias.url += ("/config/" => "XX_ALLSKY_CONFIG_XX/") alias.url += ("/website/" => "XX_ALLSKY_WEBSITE_XX/") alias.url += ("/documentation" => "XX_ALLSKY_DOCUMENTATION_XX/") alias.url += ("/overlay" => "XX_ALLSKY_OVERLAY_XX/") diff --git a/config_repo/options.json.repo b/config_repo/options.json.repo index b8116147e..411469d89 100644 --- a/config_repo/options.json.repo +++ b/config_repo/options.json.repo @@ -9,20 +9,22 @@ { "name" : "takeDaytimeImages", "default" : 1, -"description" : "Activate to take daytime images.", +"description" : "Activate to take daytime images.", "label" : "Daytime Capture", -"type" : "checkbox", +"type" : "boolean", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 1 }, { "name" : "saveDaytimeImages", "default" : 0, -"description" : "Activate to save daytime images.
Only applies if Daytime Capture is enabled.", +"description" : "Activate to save daytime images.
Only applies if Daytime Capture is enabled.", "label" : "Daytime Save", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -30,29 +32,30 @@ "default" : 1, "description" : "Activate to enable daytime auto-exposure.", "label" : "Auto-Exposure", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { "name" : "daymaxautoexposure", -"minimum" : "maxautoexposure_min", -"maximum" : "maxautoexposure_max", -"default" : "maxautoexposure_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Maximum daytime Auto-Exposure time in milliseconds (1000ms = 1 sec).
Only applies if daytime Auto-Exposure is enabled.", "label" : "Max Auto-Exposure", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 0 }, { "name" : "dayexposure", -"minimum" : "exposure_min", -"maximum" : "exposure_max", -"default" : "exposure_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "day_default", "description" : "Manual exposure time in milliseconds (1000ms = 1 sec). Can be a fraction.
If using daytime Auto-Exposure, this number is used as a starting point.", "label" : "Manual Exposure", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 1 }, @@ -60,33 +63,34 @@ "name" : "daymean", "minimum" : 0.0, "maximum" : 1.0, -"default" : 0.5, +"default" : "day_default", "description" : "The target mean brightness level when Auto-Exposure is on. 1.0 is pure white.
Best used when Auto-Exposure and Auto-Gain are on.
0 disables this auto-exposure mode.", "label" : "Mean Target", -"type" : "number", -"display" : "mean_display", +"type" : "float", +"display" : 1, "advanced" : 1 }, { "name" : "daybrightness", -"minimum" : "brightness_min", -"maximum" : "brightness_max", -"default" : "brightness_default", -"description" : "Changes the amount of light in the image.
Higher values are brighter.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", +"description" : "Deprecated. Use Mean Target instead.
Changes the amount of light in the image.
Higher numbers are brighter.", "label" : "Brightness", -"type" : "number", -"display" : "brightness_display", +"type" : "integer", +"display" : "_display", "advanced" : 0 }, { "name" : "daydelay", -"minimum" : "delay_min", +"minimum" : "_min", "maximum" : "none", "default" : 30000, "description" : "Delay between daytime images in milliseconds (1000ms = 1 sec).", "label" : "Delay", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -94,42 +98,44 @@ "default" : 0, "description" : "Activate to enable dayime Auto-Gain.", "label" : "Auto-Gain", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { "name" : "daymaxautogain", -"minimum" : "maxautogain_min", -"maximum" : "maxautogain_max", -"default" : "maxautogain_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Maximum gain when using Auto-Gain.", "label" : "Max Auto-Gain", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 1 }, { "name" : "daygain", -"minimum" : "gain_min", -"maximum" : "gain_max", -"default" : "gain_default", -"description" : "Manually sets the light sensitivity of the camera.
A high value returns a brighter image but more noise too.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "day_default", +"description" : "Manually sets the light sensitivity of the camera.
A high number returns a brighter image but more noise too. Can normally leave at 0 for daytime.", "label" : "Gain", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 1 }, { "name" : "daybin", "default" : 1, -"description" : "Binning level. 1x1 = binning OFF.", +"description" : "Binning level. 1x1 = OFF.", "label" : "Binning", "type" : "select", "options" : [ "bin_values" ], "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -137,30 +143,31 @@ "default" : 0, "description" : "Sets Auto White Balance.", "label" : "Auto White Balance", -"type" : "checkbox", -"display" : "awb_display", +"type" : "boolean", +"display" : "_display", +"generic" : 1, "advanced" : 0 }, { "name" : "daywbr", -"minimum" : "wbr_min", -"maximum" : "wbr_max", -"default" : "wbr_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Red component for the white balance.
When using Auto White Balance, this number is a starting point.", "label" : "Red Balance", -"type" : "number", -"display" : "wbr_display", +"type" : "float", +"display" : "_display", "advanced" : 0 }, { "name" : "daywbb", -"minimum" : "wbb_min", -"maximum" : "wbb_max", -"default" : "wbb_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Blue component for the white balance.
When using Auto White Balance, this number is a starting point.", "label" : "Blue Balance", -"type" : "number", -"display" : "wbb_display", +"type" : "float", +"display" : "_display", "advanced" : 0 }, { @@ -170,8 +177,9 @@ "default" : 5, "description" : "When starting Allsky during the day, skip up to this many frames while the software gets to the correct exposure.
Only applies if daytime Auto-Exposure is on.", "label" : "Frames To Skip", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -179,19 +187,21 @@ "default" : 0, "description" : "Activate to use cooling (works only on cooled cameras).", "label" : "Cooling", -"type" : "checkbox", -"display" : "EnableCooler_display", +"type" : "boolean", +"display" : "_display", +"generic" : 1, "advanced" : 1 }, { "name" : "dayTargetTemp", -"minimum" : "TargetTemp_min", -"maximum" : "TargetTemp_max", -"default" : "TargetTemp_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Sensor's target temperature when cooler is enabled (degrees Celsius).", "label" : "Target Temp.", -"type" : "number", -"display" : "TargetTemp_display", +"type" : "integer", +"display" : "_display", +"generic" : 1, "advanced" : 1 }, { @@ -200,7 +210,7 @@ "description" : "Name of the day camera tuning file to use. Omit this option to preserve the default behavior of libcamera.
Please read this documentation or this document if using a Raspberry camera with libcamera. For other Raspberry cameras read the camera manual for more information.", "label" : "Tuning File", "type" : "widetext", -"display" : "TuningFile_display", +"display" : "_display", "checkchanges" : 1, "optional" : 1, "advanced" : 1 @@ -217,29 +227,30 @@ "default" : 1, "description" : "Activate to enable nighttime auto-exposure.", "label" : "Auto-Exposure", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "nightmaxautoexposure", -"minimum" : "maxautoexposure_min", -"maximum" : "maxautoexposure_max", -"default" : "maxautoexposure_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Maximum nighttime Auto-Exposure time in milliseconds (1000ms = 1 sec).
Only applies if nighttime Auto-Exposure is enabled.", "label" : "Max Auto-Exposure", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 0 }, { "name" : "nightexposure", -"minimum" : "exposure_min", -"maximum" : "exposure_max", -"default" : "exposure_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "night_default", "description" : "Manual exposure time in milliseconds (1000ms = 1 sec). Can be a fraction.
If using nighttime Auto-Exposure, this number is used as a starting point.", "label" : "Manual Exposure", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 0 }, @@ -247,33 +258,34 @@ "name" : "nightmean", "minimum" : 0.0, "maximum" : 1.0, -"default" : 0.2, +"default" : "night_default", "description" : "The target mean brightness level when Auto-Exposure is on. 1.0 is pure white.
Best used when Auto-Exposure and Auto-Gain are on.
0 disables this auto-exposure mode.", "label" : "Mean Target", -"type" : "number", -"display" : "mean_display", +"type" : "float", +"display" : 1, "advanced" : 1 }, { "name" : "nightbrightness", -"minimum" : "brightness_min", -"maximum" : "brightness_max", -"default" : "brightness_default", -"description" : "Changes the amount of light in the image.
Higher values are brighter.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", +"description" : "Deprecated. Use Mean Target instead.
Changes the amount of light in the image.
Higher numbers are brighter.", "label" : "Brightness", -"type" : "number", -"display" : "brightness_display", +"type" : "integer", +"display" : "_display", "advanced" : 0 }, { "name" : "nightdelay", -"minimum" : "delay_min", +"minimum" : "_min", "maximum" : "none", "default" : 30000, "description" : "Delay between nighttime images in milliseconds (1000ms = 1 sec).", "label" : "Delay", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -281,42 +293,44 @@ "default" : 0, "description" : "Activate to enable nighttime Auto-Gain.", "label" : "Auto-Gain", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "nightmaxautogain", -"minimum" : "maxautogain_min", -"maximum" : "maxautogain_max", -"default" : "maxautogain_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Maximum gain when using Auto-Gain.", "label" : "Max Auto-Gain", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 1 }, { "name" : "nightgain", -"minimum" : "gain_min", -"maximum" : "gain_max", -"default" : "gain_default", -"description" : "Manually sets the light sensitivity of the camera.
A high value returns a brighter image but more noise too.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "night_default", +"description" : "Manually sets the light sensitivity of the camera.
A high number returns a brighter image but more noise too.", "label" : "Gain", -"type" : "number", +"type" : "float", "display" : 1, "advanced" : 0 }, { "name" : "nightbin", "default" : 1, -"description" : "Binning level. 1x1 = binning OFF.", +"description" : "Binning level. 1x1 = OFF.", "label" : "Binning", "type" : "select", "options" : [ "bin_values" ], "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -324,30 +338,31 @@ "default" : 0, "description" : "Activate to enable Auto White Balance (camera adjusts color).", "label" : "Auto White Balance", -"type" : "checkbox", -"display" : "awb_display", +"type" : "boolean", +"display" : "_display", +"generic" : 1, "advanced" : 0 }, { "name" : "nightwbr", -"minimum" : "wbr_min", -"maximum" : "wbr_max", -"default" : "wbr_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Red component for the white balance.
When using Auto White Balance, this number is a starting point.", "label" : "Red Balance", -"type" : "number", -"display" : "wbr_display", +"type" : "float", +"display" : "_display", "advanced" : 0 }, { "name" : "nightwbb", -"minimum" : "wbb_min", -"maximum" : "wbb_max", -"default" : "wbb_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Blue component for the white balance.
When using Auto White Balance, this number is a starting point.", "label" : "Blue Balance", -"type" : "number", -"display" : "wbb_display", +"type" : "float", +"display" : "_display", "advanced" : 0 }, { @@ -357,8 +372,9 @@ "default" : 1, "description" : "When starting Allsky at night, skip up to this many frames while the software gets to the correct exposure.
Only applies if nighttime Auto-Exposure is on.", "label" : "Frames To Skip", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -366,19 +382,21 @@ "default" : 0, "description" : "Activate to use cooling (works only on cooled cameras).", "label" : "Cooling", -"type" : "checkbox", -"display" : "EnableCooler_display", +"type" : "boolean", +"display" : "_display", +"generic" : 1, "advanced" : 1 }, { "name" : "nightTargetTemp", -"minimum" : "TargetTemp_min", -"maximum" : "TargetTemp_max", -"default" : "TargetTemp_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Sensor's target temperature when cooler is enabled (degrees Celsius).", "label" : "Target Temp.", -"type" : "number", -"display" : "TargetTemp_display", +"type" : "integer", +"display" : "_display", +"generic" : 1, "advanced" : 1 }, { @@ -387,7 +405,7 @@ "description" : "Name of the night camera tuning file to use.", "label" : "Tuning File", "type" : "widetext", -"display" : "TuningFile_display", +"display" : "_display", "checkchanges" : 1, "optional" : 1, "advanced" : 1 @@ -402,7 +420,7 @@ { "name" : "config", "default" : "[none]", -"description" : "Optional configuration file to use for settings.", +"description" : "Configuration file to use for settings.", "label" : "Configuration File", "type" : "widetext", "display" : 1, @@ -413,109 +431,111 @@ { "name" : "extraArgs", "default" : "", -"description" : "Optional extra arguments to pass to the capture program that Allsky doesn't support.", +"description" : "Extra arguments to pass to the capture program that Allsky doesn't support.", "label" : "Extra Arguments", "type" : "widetext", -"display" : "extraArgs_display", +"display" : "_display", "optional" : 1, "advanced" : 1 }, { "name" : "saturation", -"minimum" : "saturation_min", -"maximum" : "saturation_max", -"default" : "saturation_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Changes the saturation level of the image.
The lowest level produces a black and white image.", "label" : "Saturation", -"type" : "number", -"display" : "saturation_display", +"type" : "float", +"display" : "_display", "advanced" : 1 }, { "name" : "contrast", -"minimum" : "contrast_min", -"maximum" : "contrast_max", -"default" : "contrast_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Changes the contrast of an image.", "label" : "Contrast", -"type" : "number", -"display" : "contrast_display", +"type" : "float", +"display" : "_display", "advanced" : 1 }, { "name" : "sharpness", -"minimum" : "sharpness_min", -"maximum" : "sharpness_max", -"default" : "sharpness_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "Changes the sharpness of an image.
Too sharp and images look unnatural.", "label" : "Sharpness", -"type" : "number", -"display" : "sharpness_display", +"type" : "float", +"display" : "_display", "advanced" : 1 }, { "name" : "gamma", -"minimum" : "gamma_min", -"maximum" : "gamma_max", -"default" : "gamma_default", -"description" : "Changes the difference between light and dark areas.
higher values increase contrast.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", +"description" : "Changes the difference between light and dark areas.
higher numbers increase contrast.", "label" : "Gamma", -"type" : "number", -"display" : "gamma_display", +"type" : "integer", +"display" : "_display", "advanced" : 1 }, { "name" : "offset", -"minimum" : "offset_min", -"maximum" : "offset_max", -"default" : "offset_default", -"description" : "Adds about 1/10 the specified value to every pixel to brighten everything.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", +"description" : "Adds about 1/10 the specified number to every pixel to brighten everything.", "label" : "Offset", -"type" : "number", -"display" : "offset_display", +"type" : "integer", +"display" : "_display", "advanced" : 1 }, { "name" : "aggression", -"minimum" : "aggression_min", -"maximum" : "aggression_max", -"default" : "aggression_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "How much of a calculated exposure change should be made during Auto-Exposure?
Lower numbers smooth out changes but take longer to react to brightness changes.", "label" : "Aggression", -"type" : "number", -"display" : "aggression_display", +"type" : "percent", +"display" : "_display", +"generic" : 1, "advanced" : 1 }, { "name" : "gaintransitiontime", "minimum" : 0, "maximum" : "none", -"default" : "gaintransitiontime_default", +"default" : "_default", "description" : "Number of minutes over which to increase or decrease the gain when switching between day and night.
This helps smooth brightness differences. Only works if nighttime Auto-Gain is off. 0 disables transitions.", "label" : "Gain Transition Time", -"type" : "number", -"display" : "gaintransitiontime_display", +"type" : "integer", +"display" : "_display", +"generic" : 1, "advanced" : 1 }, { "name" : "width", -"minimum" : "width_min", -"maximum" : "width_max", +"minimum" : "_min", +"maximum" : "_max", "default" : 0, "description" : "Image width in pixels. 0 = max sensor width.", "label" : "Image Width", -"type" : "number", +"type" : "integer", "display" : 1, "advanced" : 0 }, { "name" : "height", -"minimum" : "height_min", -"maximum" : "height_max", +"minimum" : "_min", +"maximum" : "_max", "default" : 0, "description" : "Image height in pixels. 0 = max sensor height.", "label" : "Image Height", -"type" : "number", +"type" : "integer", "display" : 1, "advanced" : 0 }, @@ -536,21 +556,22 @@ "minimum" : 1, "maximum" : 100, "default" : 95, -"description" : "Range for a PNG image: 0-9. Range for a JPG image: 0-100.
Higher numbers produce larger, clearer files.", -"label" : "Quality", -"type" : "number", +"description" : "JPG quality: 0-100. Higher numbers produce larger, clearer files.
PNG compression: 0-9. Higher numbers produce smaller files but take longer to save.", +"label" : "Quality / Compression", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "meanthreshold", -"minimum" : "meanthreshold_min", -"maximum" : "meanthreshold_max", -"default" : "meanthreshold_default", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", "description" : "When using Mean Target this specifies how close to the target the brightness should be.", "label" : "Mean Threshold", -"type" : "number", -"display" : "meanthreshold_display", +"type" : "float", +"display" : "_display", "advanced" : 1 }, { @@ -558,29 +579,30 @@ "default" : 1, "description" : "Automatically set the USB bandwidth.", "label" : "Auto USB Bandwidth", -"type" : "checkbox", -"display" : "autousb_display", +"type" : "boolean", +"display" : "_display", "advanced" : 1 }, { "name" : "usb", -"minimum" : "usb_min", -"maximum" : "usb_max", -"default" : "usb_default", -"description" : "USB bandwidth.
If you get ASI_ERROR_TIMEOUT errors in the log file, try chaning this value.", +"minimum" : "_min", +"maximum" : "_max", +"default" : "_default", +"description" : "USB bandwidth.
If you get ASI_ERROR_TIMEOUT errors in the log file, try changing this number.", "label" : "USB Bandwidth", -"type" : "number", -"display" : "usb_display", +"type" : "integer", +"display" : "_display", "advanced" : 1 }, { "name" : "filename", "default" : "image.jpg", -"description" : "Extensions allowed: .jpg or .png.
WARNING: saving a .png file can take 10 or more seconds so be sure the time between successive images is long enough. To measure how long it takes your Pi to save a .png file, set the Debug Level to 3 and look in the log file.", +"description" : "Extensions allowed: .jpg or .png.
WARNING: saving a .png file can take 10 or more seconds so be sure the time between successive images is long enough. To measure how long it takes your Pi to save a .png file, set the Debug Level to 4 and look in the log file.", "label" : "Filename", "type" : "text", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -592,13 +614,13 @@ "options" : [ "rotation_values" ], -"display" : "rotation_display", +"display" : "_display", "advanced" : 0 }, { "name" : "flip", "default" : 0, -"description" : "Select a flipping option if you want to reverse the image along an axis.", +"description" : "Flips the image along an axis.", "label" : "Flip", "type" : "select", "options" : [ @@ -615,37 +637,51 @@ "default" : 1, "description" : "Activate to display notification images, e.g., 'Camera is off during the day'.
While these messages appear in the WebUI and Allsky Website, they are not saved so don't appear in timelapse, startrails, or keograms.", "label" : "Notification Images", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "consistentDelays", "default" : 1, -"description" : "Activate to have the delays between images be of consistent length.
The time between the start of exposures will always be (max exposure + delay).", +"description" : "Activate to have the delays between images be a consistent length.
The time between the start of exposures will always be (max exposure + delay).", "label" : "Consistent Delays Between Images", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { +"name" : "timeformat", +"default" : "%Y%m%d %H:%M:%S", +"description" : "Determines the format of the displayed time. Run 'man 3 strftime' to see the options.", +"label" : "Time Format", +"type" : "text", +"display" : 1, +"generic" : 1, +"advanced" : 0 +}, +{ "name" : "latitude", "default" : "", -"description" : "Latitude of the camera. Can be a number (-90 to 90) or a positive number with direction (N or S), for example: 60.7N.", +"description" : "Camera latitude - a number with a sign (-90 to +90) or a positive number with direction (N or S), for example: 60.7N.", "label" : "Latitude", "type" : "text", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "longitude", "default" : "", -"description" : "Longitude of the camera. Can be a number (-180 to 180) or a positive number with a direction (E or W), for example: 135.05W.", +"description" : "Camera longitude - a number with a sign (-180 to +180) or a positive number with a direction (E or W), for example: 135.05W.", "label" : "Longitude", "type" : "text", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -653,51 +689,63 @@ "default" : "-6", "description" : "Altitude of the Sun in degrees relative to the horizon at which to switch between day and night.", "label" : "Angle", -"type" : "number", +"type" : "float", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "takeDarkFrames", "default" : 0, -"description" : "Activate to capture dark frames, which are used to decrease noise. This mode hides text and time overlays.
Will continually take dark frames until restarted.", +"description" : "Activate to capture dark frames, which are used to decrease noise.
Continues taking dark frames until Allsky is restarted.", "label" : "Take Dark Frames", -"type" : "checkbox", +"type" : "boolean", "display" : 1, "advanced" : 0 }, { "name" : "useDarkFrames", "default" : 0, -"description" : "This enables dark frame subtraction, assuming you have dark frames.", +"description" : "Enables dark frame subtraction if you have dark frames.", "label" : "Use Dark Frames", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "locale", "default" : "", -"description" : "Your locale, used to determine what the thousands and decimal separators are.
Type 'locale' at a command prompt to see your choices.", +"description" : "Your locale, used to determine what the thousands and decimal separators are.
Type locale at a command prompt to see your choices.", "label" : "Locale", "type" : "text", -"display" : 0, +"display" : 1, +"generic" : 1, "advanced" : 0 }, { +"name" : "experimentalExposure", +"default" : 0, +"description" : "Activate to use a newer auto-exposure algorithm at night. Initial testing indictes the images taken during the day-to-night transition as well as at night have better exposures.
If you use this, please add a Discussion item describing your results - good or bad. We need the feedback.", +"label" : "New Exposure Algorithm", +"type" : "boolean", +"display" : "_display", +"advanced" : 1 +}, +{ "name" : "histogrambox", "default" : "500 500 50 50", "description" : "Size of the histogram box (X and Y in pixels) and offset from left and top (0-100 percent). Sizes must be even numbers.", "label" : "Histogram Box", "type" : "text", -"display" : "histogrambox_display", +"display" : "_display", "advanced" : 1 }, { "name" : "debuglevel", "default" : 1, -"description" : "Debug level. 0 is errors only.", +"description" : "Debug level. 0 is errors only. 4 is for Allsky developers use.", "label" : "Debug Level", "type" : "select", "options" : [ @@ -715,8 +763,8 @@ "default" : 1, "description" : "Activate to use the Allsky version 0.8 exposure method which decreases sensor temperature. If you see ASI_ERROR_TIMEOUTs in the log file, deactivate this.", "label" : "Version 0.8 Exposure", -"type" : "checkbox", -"display" : "newexposure_display", +"type" : "boolean", +"display" : "_display", "advanced" : 1 }, { @@ -724,8 +772,9 @@ "default" : 1, "description" : "Determines if you need to login to the WebUI or not.
If your Pi is accessible on the Internet, do NOT turn this off!!.", "label" : "Require WebUI Login", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -733,8 +782,9 @@ "default" : 0, "description" : "Activate to always show the advanced options.", "label" : "Always Show Advanced", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -747,7 +797,7 @@ { "name" : "overlayMethod", "default" : 0, -"description" : "Choose how to have image overlays done.
module has more features and is very flexible.
When set to module, the overlay settings below do NOT apply.", +"description" : "Determines how image overlays are done.
module supports a visual editor.
When set to module, the overlay settings below do NOT apply.", "label" : "Overlay Method", "type" : "select", "options" : [ @@ -756,6 +806,7 @@ ], "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -763,17 +814,9 @@ "default" : 1, "description" : "Activate to display the time an image was taken on the overlay.", "label" : "Show Time", -"type" : "checkbox", -"display" : 1, -"advanced" : 0 -}, -{ -"name" : "timeformat", -"default" : "%Y%m%d %H:%M:%S", -"description" : "Determines the format of the displayed time. Run 'man 3 strftime' to see the options.", -"label" : "Time Format", -"type" : "text", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -781,8 +824,9 @@ "default" : 1, "description" : "Activate to display the camera sensor temperature on the overlay.", "label" : "Show Temperature", -"type" : "checkbox", -"display" : "showTemp_display", +"type" : "boolean", +"display" : "_display", +"generic" : 1, "advanced" : 0 }, { @@ -797,6 +841,7 @@ {"value" : "B", "label" : "Both"} ], "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -804,8 +849,9 @@ "default" : 1, "description" : "Activate to display the exposure length on the overlay.
If Auto-Exposure is set, '(auto)' will appear after the exposure time.", "label" : "Show Exposure", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -813,26 +859,28 @@ "default" : 0, "description" : "Activate to display the camera gain on the overlay.
If Auto-Gain is set, '(auto)' will appear after the gain value.", "label" : "Show Gain", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "showBrightness", "default" : 0, -"description" : "Activate to display the Brightness value you set on the overlay.", +"description" : "Activate to display the Brightness number you set on the overlay.", "label" : "Show Brightness", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "showUSB", "default" : 0, -"description" : "Activate to display the USB Bandwidth value on the overlay.
This is primarily for troubleshooting.", +"description" : "Activate to display the USB Bandwidth number on the overlay.
This is primarily for troubleshooting.", "label" : "Show USB", -"type" : "checkbox", -"display" : "showUSB_display", +"type" : "boolean", +"display" : "_display", "advanced" : 1 }, { @@ -840,8 +888,9 @@ "default" : 0, "description" : "Activate to display the calculated mean image brightness on the overlay.", "label" : "Show Mean Brightness", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -849,8 +898,8 @@ "default" : 0, "description" : "Activate to show an outline of the histogram box, useful for determining what the 'Histogram Box' numbers should be.", "label" : "Show Histogram Box", -"type" : "checkbox", -"display" : "showhistogrambox_display", +"type" : "boolean", +"display" : "_display", "advanced" : 1 }, { @@ -858,8 +907,9 @@ "default" : 0, "description" : "Activate to display a focus metric to help you focus the camera.
This is only usefull while focusing.", "label" : "Show Focus Metric", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 1 }, { @@ -870,6 +920,7 @@ "type" : "widetext", "display" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -881,6 +932,7 @@ "display" : 1, "checkchanges" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -890,8 +942,9 @@ "default" : 0, "description" : "The maximum age of the Extra Text File in seconds.
After this time the contents of the file will no longer be displayed. Set to 0 to always display.", "label" : "Max Age Of Extra", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -899,10 +952,11 @@ "minimum" : "font height", "maximum" : "image height", "default" : 30, -"description" : "The line height of the text in pixels. If increasing the font size causes the text to overlap then increase this value.", +"description" : "The line height of the text in pixels. If increasing the font size causes the text to overlap then increase this number.", "label" : "Line Height", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -912,8 +966,9 @@ "default" : 15, "description" : "Text will begin this many pixels from the left.", "label" : "Text X", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -923,8 +978,9 @@ "default" : 35, "description" : "Text will begin this many pixels from the top.", "label" : "Text Y", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -944,25 +1000,28 @@ {"value" : 7, "label" : "Script Complex"} ], "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "fontcolor", "default" : "255 0 0", "minimum" : 0, -"description" : "Blue, Green, and Red (BGR) values of text color.
NOTE: When using RAW 16 only the first two values are used i.e., 255 128 0.", +"description" : "Blue, Green, and Red (BGR) values of text color.
NOTE: When using RAW 16 only the first two numbers are used i.e., 255 128 0.", "label" : "Font Color", "type" : "text", "display" : 1, +"generic" : 1, "advanced" : 0 }, { "name" : "smallfontcolor", "default" : "0 0 255", -"description" : "BGR value of text color.
NOTE: When using RAW 16 only the first two values are used i.e., 255 128 0.", +"description" : "BGR value of text color.
NOTE: When using RAW 16 only the first two numbers are used i.e., 255 128 0.", "label" : "Small Font Color", "type" : "text", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -977,6 +1036,7 @@ {"value" : 2, "label" : "4 connected"} ], "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -986,8 +1046,9 @@ "default" : 7, "description" : "Text Font Size. The time, if displayed, will use this size; other text displayed will be about 20% smaller.", "label" : "Font Size", -"type" : "number", +"type" : "float", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -997,8 +1058,9 @@ "default" : 1, "description" : "How thick the text line should be.", "label" : "Font Weight", -"type" : "number", +"type" : "integer", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1006,8 +1068,9 @@ "default" : 0, "description" : "Adds a black outline to the text overlay to improve contrast.", "label" : "Use Outline Font", -"type" : "checkbox", +"type" : "boolean", "display" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1022,9 +1085,10 @@ "default" : 0, "description" : "Activate to add a link to the Allsky Website to display the settings on this page.
Only applies if you have a local or remote Allsky Website.", "label" : "Display Settings", -"type" : "checkbox", +"type" : "boolean", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1032,9 +1096,10 @@ "default" : 0, "description" : "Activate to have your camera appear on the Allsky Map .", "label" : "Show on Map", -"type" : "checkbox", +"type" : "boolean", "display" : 1, "checkchanges" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1046,6 +1111,7 @@ "display" : 1, "checkchanges" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1057,6 +1123,7 @@ "display" : 1, "checkchanges" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1068,6 +1135,7 @@ "display" : 1, "checkchanges" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1079,6 +1147,7 @@ "display" : 1, "checkchanges" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1095,7 +1164,7 @@ { "name" : "lens", "default" : "", -"description" : "The lens you're using on your camera, for example: Arecont 1.55.", +"description" : "The lens you're using on your camera, for example: Arecont 1.55.", "label" : "Lens", "type" : "text", "display" : 1, @@ -1112,6 +1181,7 @@ "display" : 1, "checkchanges" : 1, "optional" : 1, +"generic" : 1, "advanced" : 0 }, { @@ -1124,12 +1194,13 @@ { "name" : "cameraType", "default" : "", -"description" : "The type of camera you are using. RPi includes the HQ and compatible cameras.", +"description" : "The type of camera you are using. Refresh re-sets the Camera Model and is useful if you change cameras of the same Camera Type.", "label" : "Camera Type", "type" : "select", "options" : [ {"value" : "RPi", "label" : "RPi"}, - {"value" : "ZWO", "label" : "ZWO"} + {"value" : "ZWO", "label" : "ZWO"}, + {"value" : "Refresh", "label" : "Refresh"} ], "display" : 1, "checkchanges" : 1, @@ -1151,8 +1222,8 @@ "default" : 0, "description" : "If multiple cameras are connected to the Pi, this is the camera number (starts at 0).", "label" : "Camera Number", -"type" : "number", -"display" : "cameraNumber_display", +"type" : "integer", +"display" : "_display", "display" : 0, "checkchanges" : 1, "advanced" : 1 diff --git a/config_repo/requirements-buster.txt b/config_repo/requirements-buster.txt index 8830128ef..8cbe2bb47 100644 --- a/config_repo/requirements-buster.txt +++ b/config_repo/requirements-buster.txt @@ -1,15 +1,15 @@ -opencv_python>=4.5.3.56 +numpy==1.21.4 +opencv-python<=4.6.0.66 Pillow pyephem skyfield astral pytz -scipy -scikit-image +scipy<=1.8.1 +scikit-image<=0.19.3 paho-mqtt -numpy>=1.21 photutils -astropy +astropy==4.3.1 suncalc Adafruit-Blinka vcgencmd diff --git a/config_repo/requirements.txt b/config_repo/requirements.txt index 42d50526d..0344b54ca 100644 --- a/config_repo/requirements.txt +++ b/config_repo/requirements.txt @@ -1,13 +1,13 @@ -opencv_python>=4.5.3.56 +numpy<=1.24.2 +opencv-python<=4.6.0.66 Pillow pyephem skyfield astral pytz -scipy -scikit-image +scipy<=1.8.1 +scikit-image<=0.19.3 paho-mqtt -numpy>=1.23.3 photutils astropy suncalc diff --git a/html/documentation/basics/Pi.html b/html/documentation/basics/Pi.html index 72470536c..a1b813525 100644 --- a/html/documentation/basics/Pi.html +++ b/html/documentation/basics/Pi.html @@ -37,34 +37,43 @@

Operating Systems

The Linux operating system controls the Pi's hardware and lets you run software such as Allsky. -The two most common versions of the Linux operating system for the Pi are -Buster which was released several years ago, and -Bullseye which was released late 2021. -Allsky runs on both operating systems. -Most people are running the 32-bit version of the operating systems, -although some people have the 64-bit version of Bullseye. +The two most common versions of Linux for the Pi are Buster, +which was released several years ago, +and Bullseye which was released late 2021. +Allsky runs on both operating systems but use Bullseye if possible, +especially if you have an RPi camera. +Allsky works on both the 32-bit and 64-bit versions of Bullseye; +there was no official 64-bit version of Buster and Allsky is not supported on it.

-There are also different distributions of Linux (called "distros") from various companies. +There are different distributions of Linux (called "distros") from various companies. While they are basically the same Linux, there can be subtle differences and the administrative commands can vary. -The recommended Linux distro is Debian.

+
+The recommended Linux distro is Debian. +We can't guarantee Allsky will run on other distros. +

Raspberry Pi Models

-There are several models of the Pi including 1, 2, 3, 4, Zero, and others. +There are several models of the Pi including Zero, Zero 2, 1, 2, 3, 4, and others. The Pi 3 and 4 are the most common being used for Allsky, -but some people have the Pi Zero because of its low cost. +but some people have the Pi Zero or Zero 2 because of their low cost. The various models have different processing power; -the Pi 4 is the most powerful, followed by the 3, then 2, then 1. -The Zero is somewhere around the Pi 3 in terms of processing power. -The Pi 4 is available with up to 8 GB memory whereas the Pi Zero is limited to 0.5 GB memory. -Although Allsky can run on the Zero, several GB of swap space are needed, +the Pi 4 is the most powerful, followed by the 3, then 2, then 1, then Zero. +The Zero 2 is near the Pi 3 in terms of processing power but has less memory. +The Pi 4 is available with up to 8 GB memory whereas the +Pi Zero and Zero 2 are limited to 0.5 GB memory. +Although Allsky can run on the Zero 2, several GB of swap space are needed, and even then, Allsky just barely runs. -The ideal unit is a Pi 4 with 4 GB memory. +The processing power of the Pi Zero is very limited so you likely won't +be able to create timelapse videos - stay away from it if possible. +

+

+The ideal unit is a Pi 4 with 4 GB memory. Allsky runs nicely on that and the USB 3.0 ports are nice for cameras that support it.

@@ -73,7 +82,7 @@

Hardware Overview

Newer devices come with USB 2 ports and the Pi 4 has a couple USB 3 ports. -They also have a special port to plug the RPi High Definition (HD) camera into. +They also have a special port to plug RPi and compatible cameras into.

diff --git a/html/documentation/changeLog.html b/html/documentation/changeLog.html index 5e192268e..ec9fdb237 100644 --- a/html/documentation/changeLog.html +++ b/html/documentation/changeLog.html @@ -1,5 +1,6 @@ + @@ -19,497 +20,718 @@ Allsky Version Change Log + - -
- -
- -

-This page lists the changes to all the major Allsky releases. -You can also see the Known Issues and Limitations -of the current release. -

- -
-Wonder what all the colors below mean? -Check out this page. -
- -

v2023.05.01 - Current Release

-
- -

Core Allsky

-
    -
  • New camera support: -
      -
    • All ZWO cameras as of May 1, 2023. -
    • RPi HQ and Module 3 cameras. -
    • ArduCam 16 and 64 MP cameras. -
    • The RPi "Global Shutter Camera" is NOT supported - high speed shutters aren't useful for allsky images. -
    -
  • "Mini" timelapse videos can be created that contain a user-configurable - number of the most recent images. - This allows you to continually see the recent sky conditions. -
  • Installation improvements: -
      -
    • If there is not enough swap space configured you are prompted to add more. - Doing this decreases the chance of timelapse creation problems. -
    • If allsky/tmp is not a memory-resident - filesystem you are prompted to make it one. - This SIGNIFICANTLY decreases the number of writes to the SD card, prolonging its life. -
    • If a ~/allsky-OLD directory is found it's - assumed to be a prior release of Allsky and you are prompted to have its images, - darks, and other items moved to the new release. - See the Installing / Upgrading -> Allsky - page for instructions for installing this release. -
    -
  • scripts/check_allsky.sh was added to perform basic sanity - checking of your Allsky installation. - Run it after you're done configuring Allsky to see if you have any issues. -
  • latitude and longitude - can now be specified as either a decimal number - (e.g., -105.21) or with N, S, E, W (e.g., 105.21W). -
  • Removed several settings from config.sh: -
      -
    • CAMERA: To update the camera type, use the new - Camera Type setting in the WebUI. - This is an advanced setting so you need to click the "Show Advanced Options" button to view it. -
    • POST_END_OF_NIGHT_DATA is no longer needed since - Allsky automatically determines if you have a local Allsky Website, a remote one, or both. -
    -
  • New ftp-settings.sh variables: -
      -
    • REMOTE_PORT specifies a non-default FTP port. -
    • SSH_KEY_FILE is the path to an SSH private key. - When scp is used for uploads, - this identify file will be used to establish the secure connection. -
    • The Secure CP (scp) and Google Cloud Service - (gcs) protocols are now supported for file uploads. -
    -
  • The Wiki now points to files in the GitHub documentation directory. - A copy of that directory is also on the Pi and accessible via the Documentation link in the WebUI. -
  • The Allsky Documentation has been significantly enhanced and expanded. - Its goal is to be a single source for everything you need to know about Allsky. - If you don't know how to do something, look it up. - If it's not in the documentation, let us know. -
  • AUTO_STRETCH now works, and is documented with sample images. -
  • Images can now be uploaded using the full image-YYYYMMDDHHMMSS.jpg - name instead of the shorter image.jpg name. - See the IMG_UPLOAD_ORIGINAL_NAME Allsky setting in the documentation. -
  • Many minor enhancements and bug fixes were made. -
- - -

WebUI

-
    -
  • The WebUI is now installed as part of the Allsky installation and must be - used to make all settings changes. -
    - The allsky-portal - repository will be removed as it is no longer needed. -
    -
  • New links on the left side of the WebUI: -
      -
    • Overlay Editor allows you to drag and drop the text - and images you want overlayed on the images. - This is a significant improvement over the old mechanism and lets you - vary the font size, color, rotation, etc. for everything you add. - You can use variables in the text which get replaced at run-time, e.g., the time the image was taken. -
    • Module Manager allows you to specify what actions should - take place after an image has been saved. - For example you can add an overlay or count the number of stars or periodically control a dew heater. - Users can develop (and hopefully share) their own modules. - Full notes on how to develop modules - is included in the documentation. -
    • The Allsky Documentation - link accesses the documentation on your Pi. -
    -
  • Minimum, maximum, and default values are now correct for all camera models. -
  • Required fields with missing data are shown in red with a message saying the data is missing. - For example, Latitude is a required field. -
  • New settings on the Allsky Settings page: -
      -
    • Camera Type is either ZWO or RPi. - This replaces the CAMERA variable in the - config.sh - file and also allows you to switch between cameras connected to the Pi. - For example, if you have both an RPi and ZWO camera attached, you can switch between them using this setting. -
    • Max Auto-Exposure for day and night. - When using auto-exposure, exposure times will not exceed this value. -
    • Max Auto-Gain for day and night. - When using auto-gain, gain values will not exceed this value. -
    • Auto White Balance, Red Balance, - and Blue Balance are now available for day and night. -
    • Frames to Skip for day and night determine how many initial - auto-exposure frames to ignore when starting Allsky, while the auto-exposure - algorithm homes in on the correct exposure. - These frames are often over or under exposed so not worth saving anyhow. -
    • Consistent Delays determines whether or not the time between the start of - exposures will be consistent (current behavior) or not. - When enabled, the time between images is the maximum exposure time plus the delay you set. -
    • Overlay Method determines if the text overlay (exposure, time, etc.) - should be done by the legacy program or by the new "module" system (see above). -
      The default method will change to the module method in the next release of Allsky, - and after that the legacy overlay method will be removed. -
      -
    • Require WebUI Login specifies whether - or not the WebUI should require you to login. - Only set this to No - if your Pi is on a local network and you trust everyone on the network. - Do NOT disable it if your Pi is accessible via the Internet! -
    • Cooling and Target Temp. - (ZWO only) now have separate settings for day and night. -
    • Aggression (ZWO only) - determines how much of a calculated exposure change should be applied. - This helps smooth out brightness changes, for example, when a car's headlights appear in one frame. -
    • Gamma (ZWO only) changes the contrast of an image. - It is only supported by a few cameras; for those that don't, - the AUTO_STRETCH setting can produce a similar effect. -
    • Offset (ZWO only) - adds about 1/10th the specified amount to each pixel's brightness, - thereby brightening the whole image. - etting this too high causes the image to turn gray. -
    • Contrast and - Sharpness(RPi only). -
    • Extra Parameters (RPi only) replaces the - CAPTURE_EXTRA_PARAMETERS - variable in the config.sh file, - and allows you to pass parameters to the libcamera-still - image capture program that Allsky doesn't natively support, such as auto-focus options. -
    • Mean Target (RPi only) for day and night. - This specifies the mean target brightness (0.0 (pure black) to - 1.0 (pure white)) when in auto-exposure mode. -
    • Mean Threshold (RPi only). - This specifies how close the actual mean brightness must be to the - Mean Target. - For example, if Mean Target is 0.5 and - Mean Threshold is 0.1, - the actual mean can vary between 0.4 and 0.6 (0.5 +/- 0.1). -
    • The Focus Metric setting is now available for ZWO cameras. - Higher numbers indicate better focus. - Use only when sky conditions are NOT changing. -
    -
  • NOTE: the following settings moved from - config.sh to the WebUI, - and are "advanced" options so you'll need to click the "Show Advanced Options" button to see them: -
      -
    • DAYTIME_CAPTURE is now - Take Daytime Images in the WebUI. -
    • DAYTIME_SAVE is - Save Daytime Images. -
    • DARK_CAPTURE is - Take Dark Frames. -
    • DARK_FRAME_SUBTRACTION is - Use Dark Frames. -
    -
  • Debug Level is more consistent: -
      -
    • 0: errors only. -
    • 1: level 0 plus warnings and messages about taking and saving pictures. This is the default. -
    • 2: level 1 plus details on images captured, sleep messages and the like. -
    • 3: level 2 plus time to save image, details on exposure settings and capture retries, and module execution. -
    • 4: lots of gory details for developers only. -
    -
  • System messages appear at the top of the WebUI whenever you need to take an action. -
  • Many minor enhancements were made. -
- - -

Allsky Website

-
    -
  • The Allsky Website is now installed in ~/allsky/html/allsky. -
  • If an older version of the Website is found during Website installation you'll be prompted - to have its images and settings moved to the new location. -
  • The home page can be customized: -
      -
    • You can specify the order, contents, look, and style of the icons on the left side. - You can also hide an icon or display a new one. -
    • You can specify the order, contents, and style of the popout that appears on the right side. - For example, you can add a link to pictures of your allsky camera. -
    • You can set a background image. -
    • You can easily change the maximum width of the image. -
    • You can add a link to a personal website. - This link appears at the top of the page. -
    • You can add a border around the image to have it stand out on the page. -
    • You can hide the "Make Your Own" link on the bottom right of the page. -
    • You can change the icon that appears on the browser's tab. -
    • See the Allsky Website documentation for other customizations you can make. -
    -
  • Left sidebar: -
      -
    • The constellation overlay icon (Casseopeia icon) is hidden by default and should only be displayed - after you've set the overlay to match your stars. -
    • If you are creating mini-timelapse videos, when you install the Website an icon for the current video will appear on the left side. - You can also manually show/hide the icon. -
    • There's a new icon to display the image full-size. -
    • The startrails and information icons were updated. -
    -
  • Popout on right side: -
      -
    • A link to your Image Settings can optionally be displayed via the - Display Settings option in the WebUI. -
    • The version of Allsky and the Allsky Website are displayed. -
    -
  • Timelapse video thumbnails are now created by default on the Pi and uploaded to a remote server. - This resolves issues with remote servers that don't support creating thumbnails. - See the TIMELAPSE_UPLOAD_THUMBNAIL setting. -
  • Configuration file changes: -
      -
    • The two prior configuration files (config.js and - virtualsky.json) are replaced by configuration.json. -
    • There are several new settings, including the ability to specify the opacity of the overlay. -
    • The overlaySize setting, - which defined both the width and the height of the constellation overlay, was split into - overlayWidth and overlayHeight. - Having separate values can be helpful when trying to get the overlay to line up with the actual stars. -
    • The WebUI Editor page must be used to edit the Allsky Website's - configuration file since it performs various checks before updating the configuration. -
    • The Editor page should also be used to edit a REMOTE Allsky Website's configuration file for the same reason.s - A master copy of the remote server's configuration.json - is kept on the Pi and automatically re-uploaded to the server after every change. - After you do this, the drop-down list on the Editor page will now have - configuration.json (remote Allsky Website) to distinguish it from a local Website's file. - See the Allsky Website Installation documentation for details. -
    -
- -
-
- -

v2022.03.01

-
-
    -
  • Switched to date-based release names. -
  • Added ability to have your allsky camera added to the - Allsky Map by configuring - these settings. - Added Allsky Map Setup section to the WebUI to configure the map settings. - The Lens field now shows in the popout on the Allsky Website (if installed). -
  • Significantly enhanced Wiki - more pages and more information on existing pages. - All known issues are described there as well as fixes / workarounds. -
  • Added an option to keograms to make them full width, even if few images were used in creating the keogram. - In config.sh, set KEOGRAM_EXTRA_PARAMETERS="--image-expand". -
  • Added/changed/deleted settings (in config.sh unless otherwise noted): -
      -
    • Added WEBUI_DATA_FILES - contains the name of one or more files that contain information to - be added to the WebUI's System page. - See this Wiki page for more information. -
    • Renamed NIGHTS_TO_KEEP to DAYS_TO_KEEP since it determines - how many days of data to keep, not just nighttime data. - If blank (""), ALL days' data are kept. -
    • Deleted AUTO_DELETE - its functionality is now in DAYS_TO_KEEP. - DAYS_TO_KEEP="" is similar to the old AUTO_DELETE=false. -
    • Added WEB_DAYS_TO_KEEP - specifies how many days of Allsky Website images and - videos to keep, if the website is installed on your Pi. -
    • Added WEB_IMAGE_DIR in ftp-settings.sh to allow the - images to be copied to a location on your Pi (usually the Allsky Website) as well as being copied to a remote machine. - This functionality already existed with timelapse, startrails, and keogram files. -
    -
  • The RPi camera now supports all the text overlay features as the ZWO camera, - including the Extra Text file. -
  • Removed the harmless "deprecated pixel format used" message from the timelapse log file. - That message only confused people. -
  • Improved the auto-exposure for RPi cameras. -
  • Made numerous changes to the ZWO and RPi camera's code that will make it easier to maintain and add new features in the future. -
  • If Allsky is stopped or restarted while a file is being uploaded to a remote server, - the upload continues, eliminating cases where a temporary file would be left on the server. -
  • Decreased other cases where temporary files would be left on remote servers during uploads. - Also, uploads now perform additional error checking to help in debugging. -
  • Only one upload can now be done at a time. - Any additional uploads display a message in the log file and then exit. - This should eliminate (or signifiantly decrease) cases where a file is overwritten or not found, - resulting in an error message or a temporary file left on the server. -
  • Added a --debug option to allsky/scripts/upload.sh to aid in debugging uploads. -
  • Upload log files are only created if there was an error; this saves writes to SD cards. -
  • The removeBadImages.sh script also only creates a log file if there was an error, - which saves one write to the SD card for every image. -
  • Allsky now stops with an error message on unrecoverable errors (e.g., no camera found). - It used to keep restarting and failing forever. -
  • More meaningful messages are displayed as images. - For example, in most cases "ERROR. See /var/log/allsky.log" messages have been replaced - with messages containing additional information, for example, - "*** ERROR *** Allsky Stopped! ZWO camera not found!". -
  • If Allsky is restarted, a new "Allsky software is restarting" message is displayed, - instead of a "stopping" followed by "starting" message. -
  • The timelapse debug output no longer includes one line for each of several thousand images proced. - This make it easier to see any actual errors. -
  • The Camera Settings page of the WebUI now displays the minimum, maximum, - and default values in a popup for numerical fields. -
  • Startrails and Keogram creation no longer crash if invalid files are found. -
  • Removed the allsky/scripts/filename.sh file. -
  • The RPi Gamma value in the WebUI was renamed to Saturation, - which is what it always adjusted; Gamma was incorrect. -
  • Known issues: -
      -
    • The startrails and keogram programs don't work well if you bin differently during the day and night. - If you don't save daytime images this won't be a problem. -
    • The minimum, maximum, and default values in the Camera Settings page of the WebUI, - especially for the RPi camera, aren't always correct. - This is especially try if running on the Bullseye operating system, where many of the settings changed. -
    -
-
- - -

0.8.3

-
-
    -
  • Works on Bullseye operating system. -
  • RPi version: -
      -
    • Has an improved auto-exposure algorithm. - To use it, set CAPTURE_EXTRA_PARAMETERS="-daymean 0.5 -nightmean 0.2" - in config.sh. -
    • Has many new settings including support for most of the text overlay features that are supported by the ZWO version. -
    -
  • New and changed config.sh variables, - see the Software Settings Wiki page for more information: -
      -
    • IMG_UPLOAD_FREQUENCY specifies how often the image should be uploaded to a website. - Useful with slow uplinks or metered Internet connections. -
    • IMG_CREATE_THUMBNAILS specifies whether or not thumbnails should be created for each image. -
    • REMOVE_BAD_IMAGES now defaults to "true" since bad-image detection is now done after a - picture is saved rather than once for all pictures at the end of the night. - This helps decrease problems when creating startrails, keograms, and timelapse videos. -
    • IMG_PREFIX: no longer used - the name of the image used by the websites is now - whatever you specify in the WebUI (default: image.jpg). -
    • When upgrading to 0.8.3 you MUST follow the steps listed - here. -
      -
    -
  • Replaced saveImageDay.sh and saveImageNight.sh with - saveImage.sh that has improved functionality, - including passing the sensor temperature to the dark subtraction commands, - thereby eliminating the need for the temperature.txt file. -
  • The image used by the websites (default: image.jpg) - as well as all temporary files are now written to allsky/tmp. -
    If you are using the Allsky Website you will need to change the - imageName variable in - /var/www/html/allsky/config.js to - "/current/tmp/image.jpg".
    -
  • You can significanly reduce wear on your SD card by making allsky/tmp - a memory-based filesystem. -
-
- - -

0.8.1

-
-
    -
  • Rearranged the directory structure. -
  • Created a Wiki with additional documentation and troubleshooting tips. -
  • Renamed several variables in config.sh and ftp-settings.sh. -
  • CAMERA type of "auto" is no longer supported - you must specify "ZWO" or "RPi". -
  • Startrails and keograms are now created using all CPUs on the Pi, drastically speeding up creation time. -
  • Installing the WebUI now preserves any website files (keograms, startrails, etc.) you have. - This allows for non-destructive updates of the WebUI. -
  • New script called upload.sh centralizes all the upload code from other scripts, - and can be used to debug uploading issues. - See this Wiki page for more information. -
  • The RPi camera does much better auto-exposure if you set the -mode-mean and -autoexposure options. -
  • The WebUI will now show the Pi's throttle and low-voltage states, which is useful for debugging. -
  • Darks work better. -
  • Many bug fixes, error checks, and warnings added. -
-
- - -

0.8

-
-
    -
  • Workaround for ZWO daytime autoexposure bug. -
  • Improved exposure transitions between day and night so there's not such a huge change in brightness. -
  • Decrease in ZWO sensor temperature. -
  • Lots of new settings, including splitting some settings into day and night versions. -
  • Error checking and associated log messages added in many places to aid in debugging. -
  • Ability to have "notification" images displayed, such as "Allsky is starting up" and "Taking dark frames". -
  • Ability to resize uploaded images. -
  • Ability to set thumbnail size. -
  • Ability to delete bad images (corrupt and too light/dark). -
  • Ability to set an image file name prefix. -
  • Ability to reset USB bus if ZWO camera isn't found (requires uhubctl command to be installed). -
  • Ability to specify the format of the time displayed on images. -
  • Ability to have the temperature displayed in Celcius, Fahrenheit, or both. -
  • Ability to set bitrate on timelapse video. -
-
- - -

0.7

-
-
    -
  • Added Raspberry Pi camera HQ support based on Rob Musquetier's fork. -
  • Added support for x86 architecture (Ubuntu, etc.). -
  • Temperature dependant dark frame library. -
  • Added browser-based script editor. -
  • Added configuration variables to crop black area around image. -
  • Added timelapse frame rate setting. -
  • Changed font size default value. -
-
- - -

0.6

-
-
    -
  • Added daytime exposure and auto-exposure capability. -
  • Added -maxexposure, -autoexposure, -maxgain, -autogain options. - Note that using autoexposure and autogain at the same time may produce unexpected results (black frames). -
  • Autostart is now based on systemd and should work on all raspbian based systems, including headless distributions. - Remote controlling will not start multiple instances of the software. -
  • Replaced nodisplay option with preview argument. - No preview in autostart mode. -
  • When using the WebUI, camera options can be saved without rebooting the RPi. -
  • Added a publicly accessible preview to the WebUI: public.php. -
  • Changed exposure unit to milliseconds instead of microseconds. -
-
- - -

0.5

-
-
    -
  • Added Startrails (image stacking) with brightness control. -
  • Keograms and Startrails generation is now much faster thanks to a rewrite by Jarno Paananen.. -
-
- - -

0.4

-
-
    -
  • Added Keograms (summary of the night in one image). -
-
- - -

0.3

-
-
    -
  • Added dark frame subtraction. -
-
- - -

0.2

-
-
    -
  • Separated camera settings from code logic. -
-
- - -

0.1

-
-
    -
  • Initial release. -
-
- - -
-
+ +
+ +
+

+ This page lists the changes to all Allsky releases. + You can also see the Known Issues and + Limitations + of the current release. +

+ +
+ Wonder what all the colors below mean? + Check out this page. +
+ +

v2023.05.01_01 - Point Release # 1

+
+ +

Enhancements

+
    +
  • If multiple consecutive "bad" images are found, + a warning image is now displayed saying how many "bad" images were found. + A system warning in the WebUI is also displayed with instructions on what to check. +
    + When Allsky starts it displays an "Allsky is Starting" image until the first + good image is saved. + Depending on your settings and the sky brightness, + it may take many images before it gets a good one to save. + The new warning image makes it obvious Allsky hasn't hung. +
  • +
  • In the WebUI's Editor page, + the buttons (e.g., "Save changes") are now at the top of the page + and stay there as you scroll down. + A new "Top" button appears at the bottom of the page after you scroll + to make it quick to get back to the top. +
  • +
  • Selecting Refresh from the + Camera Type drop-down + can be used to change cameras of the same + Camera Type. + For example, replacing a ZWO ASI120 with an ASI290. +
  • +
  • Greatly sped up the installation process when packages are not + already installed by using prebuilt binaries where possible. +
  • +
  • Changing to a camera that Allsky hasn't seen before will use the + prior camera's settings that typically are the same for all cameras, + for example, + Latitude, + Angle, + and + Owner. + You'll still need to verify other settings like exposure times + are appropriate for the new camera. +
    + Changing to a camera that Allsky HAS seen before will + use all that camera's prior settings. +
  • +
  • The installation only prompts for a new hostname if the current one + is still the default "raspberrypi". + The suggested new name is still "allsky". + If the current hostname is anything other than the default the user + already changed the Pi's name, so don't prompt to change again. +
  • +
  • Check at startup if the settings file is properly linked to a + camera-specific file. + When it isn't, changes to the settings won't be propogated during + updates to Allsky or when switching cameras. +
  • +
  • ZWO only: +
      +
    • ZWO Experimental Exposure + is a new setting that will use the Allsky auto-exposure algorithm + at night (assuming nighttime auto-exposure is enabled). + Initial testing indicates this improves the exposure of nightime images. +
      + If you enable this setting, please let us know if it helps or not + by entering a Discussion item in GitHub. + We need the feedback. +
      +
    • Target Mean can now be specified + for daytime and nighttime. + This is the same as for RPi cameras - values range from + 0.0 (pure black) to 1.0 (pure white). + 1.0 is equivalent to 255 in the ${MEAN} + variable in the Overlay Editor. +
    • +
    • Mean Threshold can now be specified. + This is the same as for RPi cameras and determines how close the + mean brightness needs to be to the + Target Mean. +
    • +
    +
  • +
  • If there are multiple Wi-Fi interfaces, information on each one will be displayed. +
  • +
  • The WebUI now validates numbers. + For example, entering X300 + for an exposure will produce an error. + Hovering your mouse over a number's value will tell you if it + must be a whole number or it accepts fractions. +
  • +
+ +

Bug Fixes

+
    +
  • REMOVE_BAD_IMAGES would often flag dark, + but "good" images as "bad" and not save them using the default + REMOVE_BAD_IMAGES_THRESHOLD_LOW + value of 1 (which is still a good default). +
    + If after installing this point release you are still getting a lot of "bad" images, + do NOT disable REMOVE_BAD_IMAGES - instead, + modify the REMOVE_BAD_IMAGES_THRESHOLD_* values as needed. +
  • + +
  • The s3 upload protocol now sets the destination file name to + image.jpg instead of + image-YYYYMMDDHHMMSS.jpg. +
  • +
  • The scp upload protocol now uses the + REMOTE_USER setting. +
  • +
  • Set the upper limits on the x and y spinner controls in the overlay property editors to the size + of the image, previously these were hard coded to 2048.
  • +
  • When adding a new variable to the overlay variable list it can now be selected without + refreshing the page.
  • +
  • When an image is moved out of bounds using the x and y spinner controls a red rectangle will be + drawn around the image, the same as if it were dragged out of bounds.
  • +
  • Fixed a bug when compiling on i386 platforms, due to a recent Pi OS change.
  • + +
+ +
+
+ +

v2023.05.01 - Current Major Release

+
+ + +

Core Allsky

+
    +
  • New camera support: +
      +
    • All ZWO cameras as of May 1, 2023. +
    • RPi HQ and Module 3 cameras. +
    • ArduCam 16 and 64 MP cameras. +
    • The RPi "Global Shutter Camera" is NOT supported - high speed shutters aren't useful for + allsky images. +
    +
  • "Mini" timelapse videos can be created that contain a user-configurable + number of the most recent images. + This allows you to continually see the recent sky conditions. +
  • Installation improvements: +
      +
    • If there is not enough swap space configured you are prompted to add more. + Doing this decreases the chance of timelapse creation problems. +
    • If allsky/tmp is not a memory-resident + filesystem you are prompted to make it one. + This SIGNIFICANTLY decreases the number of writes to the SD card, prolonging its life. +
    • If a ~/allsky-OLD directory is found it's + assumed to be a prior release of Allsky and you are prompted to have its images, + darks, and other items moved to the new release. + See the Installing / + Upgrading -> Allsky + page for instructions for installing this release. +
    +
  • scripts/check_allsky.sh was added to perform basic sanity + checking of your Allsky installation. + Run it after you're done configuring Allsky to see if you have any issues. +
  • latitude and longitude + can now be specified as either a decimal number + (e.g., -105.21) or with N, S, E, W (e.g., 105.21W). +
  • Removed several settings from config.sh: +
      +
    • CAMERA: To update the camera type, use the new + Camera Type setting in the WebUI. + This is an advanced setting so you need to click the "Show Advanced Options" button to + view it. +
    • POST_END_OF_NIGHT_DATA is no longer needed since + Allsky automatically determines if you have a local Allsky Website, a remote one, or + both. +
    +
  • New ftp-settings.sh variables: +
      +
    • REMOTE_PORT specifies a non-default FTP port. +
    • SSH_KEY_FILE is the path to an SSH private key. + When scp is used for uploads, + this identify file will be used to establish the secure connection. +
    • The Secure CP (scp) and Google Cloud Service + (gcs) protocols are now supported for file uploads. +
    +
  • The Wiki now points to files in the GitHub documentation + directory. + A copy of that directory is also on the Pi and accessible via the Documentation link in the + WebUI. +
  • The Allsky Documentation has been significantly enhanced and expanded. + Its goal is to be a single source for everything you need to know about Allsky. + If you don't know how to do something, look it up. + If it's not in the documentation, let us know. +
  • AUTO_STRETCH now works, and is documented with sample images. +
  • Images can now be uploaded using the full image-YYYYMMDDHHMMSS.jpg + name instead of the shorter image.jpg name. + See the IMG_UPLOAD_ORIGINAL_NAME Allsky setting in the + documentation. +
  • Many minor enhancements and bug fixes were made. +
+ + +

WebUI

+
    +
  • The WebUI is now installed as part of the Allsky installation and must be + used to make all settings changes. +
    + The allsky-portal + repository will be removed as it is no longer needed. +
    +
  • New links on the left side of the WebUI: +
      +
    • Overlay Editor allows you to drag and drop the text + and images you want overlayed on the images. + This is a significant improvement over the old mechanism and lets you + vary the font size, color, rotation, etc. for everything you add. + You can use variables in the text which get replaced at run-time, e.g., the time the + image was taken. +
    • Module Manager allows you to specify what actions should + take place after an image has been saved. + For example you can add an overlay or count the number of stars or periodically control + a dew heater. + Users can develop (and hopefully share) their own modules. + Full notes on how to develop modules + is included in the documentation. +
    • The Allsky Documentation + link accesses the documentation on your Pi. +
    +
  • Minimum, maximum, and default values are now correct for all camera models. +
  • Required fields with missing data are shown in red with a message saying the data is missing. + For example, Latitude is a required field. +
  • New settings on the Allsky Settings page: +
      +
    • Camera Type is either ZWO or RPi. + This replaces the CAMERA variable in the + config.sh + file and also allows you to switch between cameras connected to the Pi. + For example, if you have both an RPi and ZWO camera attached, you can switch between + them using this setting. +
    • Max Auto-Exposure for day and night. + When using auto-exposure, exposure times will not exceed this value. +
    • Max Auto-Gain for day and night. + When using auto-gain, gain values will not exceed this value. +
    • Auto White Balance, Red + Balance, + and Blue Balance are now available for day and night. +
    • Frames to Skip for day and night determine how many + initial + auto-exposure frames to ignore when starting Allsky, while the auto-exposure + algorithm homes in on the correct exposure. + These frames are often over or under exposed so not worth saving anyhow. +
    • Consistent Delays determines whether or not the time + between the start of + exposures will be consistent (current behavior) or not. + When enabled, the time between images is the maximum exposure time plus the delay you + set. +
    • Overlay Method determines if the text overlay + (exposure, time, etc.) + should be done by the legacy program or by the new "module" system (see above). +
      The default method will change to the module method in the next release of + Allsky, + and after that the legacy overlay method will be removed. +
      +
    • Require WebUI Login specifies whether + or not the WebUI should require you to login. + Only set this to No + if your Pi is on a local network and you trust everyone on the network. + Do NOT disable it if your Pi is accessible via the Internet! +
    • Cooling and Target + Temp. + (ZWO only) now have separate settings for day and night. +
    • Aggression (ZWO only) + determines how much of a calculated exposure change should be applied. + This helps smooth out brightness changes, for example, when a car's headlights appear in + one frame. +
    • Gamma (ZWO only) changes the contrast of an image. + It is only supported by a few cameras; for those that don't, + the AUTO_STRETCH setting can produce a similar effect. +
    • Offset (ZWO only) + adds about 1/10th the specified amount to each pixel's brightness, + thereby brightening the whole image. + etting this too high causes the image to turn gray. +
    • Contrast and + Sharpness(RPi only). +
    • Extra Parameters (RPi only) replaces the + CAPTURE_EXTRA_PARAMETERS + variable in the config.sh file, + and allows you to pass parameters to the libcamera-still + image capture program that Allsky doesn't natively support, such as auto-focus options. +
    • Mean Target (RPi only) for day and night. + This specifies the mean target brightness (0.0 (pure black) to + 1.0 (pure white)) when in auto-exposure mode. +
    • Mean Threshold (RPi only). + This specifies how close the actual mean brightness must be to the + Mean Target. + For example, if Mean Target is 0.5 and + Mean Threshold is 0.1, + the actual mean can vary between 0.4 and 0.6 (0.5 +/- 0.1). +
    • The Focus Metric setting is now available for ZWO + cameras. + Higher numbers indicate better focus. + Use only when sky conditions are NOT changing. +
    +
  • NOTE: the following settings moved from + config.sh to the WebUI, + and are "advanced" options so you'll need to click the "Show Advanced Options" button to see + them: +
      +
    • DAYTIME_CAPTURE is now + Take Daytime Images in the WebUI. +
    • DAYTIME_SAVE is + Save Daytime Images. +
    • DARK_CAPTURE is + Take Dark Frames. +
    • DARK_FRAME_SUBTRACTION is + Use Dark Frames. +
    +
  • Debug Level is more consistent: +
      +
    • 0: errors only. +
    • 1: level 0 plus warnings and messages about taking and saving pictures. This is the + default. +
    • 2: level 1 plus details on images captured, sleep messages and the like. +
    • 3: level 2 plus time to save image, details on exposure settings and capture retries, + and module execution. +
    • 4: lots of gory details for developers only. +
    +
  • System messages appear at the top of the WebUI whenever you need to take an action. +
  • Many minor enhancements were made. +
+ + +

Allsky Website

+
    +
  • The Allsky Website is now installed in ~/allsky/html/allsky. +
  • If an older version of the Website is found during Website installation you'll be prompted + to have its images and settings moved to the new location. +
  • The home page can be customized: +
      +
    • You can specify the order, contents, look, and style of the icons on the left side. + You can also hide an icon or display a new one. +
    • You can specify the order, contents, and style of the popout that appears on the right + side. + For example, you can add a link to pictures of your allsky camera. +
    • You can set a background image. +
    • You can easily change the maximum width of the image. +
    • You can add a link to a personal website. + This link appears at the top of the page. +
    • You can add a border around the image to have it stand out on the page. +
    • You can hide the "Make Your Own" link on the bottom right of the page. +
    • You can change the icon that appears on the browser's tab. +
    • See the Allsky Website documentation for other customizations you can make. +
    +
  • Left sidebar: +
      +
    • The constellation overlay icon (Casseopeia icon) is hidden by default and should only + be displayed + after you've set the overlay to match your stars. +
    • If you are creating mini-timelapse videos, when you install the Website an icon for the + current video will appear on the left side. + You can also manually show/hide the icon. +
    • There's a new icon to display the image full-size. +
    • The startrails and information icons were updated. +
    +
  • Popout on right side: +
      +
    • A link to your Image Settings can optionally be displayed via the + Display Settings option in the WebUI. +
    • The version of Allsky and the Allsky Website are displayed. +
    +
  • Timelapse video thumbnails are now created by default on the Pi and uploaded to a remote + server. + This resolves issues with remote servers that don't support creating thumbnails. + See the TIMELAPSE_UPLOAD_THUMBNAIL setting. +
  • Configuration file changes: +
      +
    • The two prior configuration files (config.js and + virtualsky.json) are replaced by configuration.json. +
    • There are several new settings, including the ability to specify the opacity of the + overlay. +
    • The overlaySize setting, + which defined both the width and the height of the constellation overlay, was split into + overlayWidth and overlayHeight. + Having separate values can be helpful when trying to get the overlay to line up with the + actual stars. +
    • The WebUI Editor page must be used to edit the Allsky + Website's + configuration file since it performs various checks before updating the configuration. +
    • The Editor page should also be used to edit a REMOTE + Allsky Website's configuration file for the same reason.s + A master copy of the remote server's configuration.json + is kept on the Pi and automatically re-uploaded to the server after every change. + After you do this, the drop-down list on the Editor page + will now have + configuration.json (remote Allsky Website) to distinguish + it from a local Website's file. + See the Allsky Website Installation documentation for details. +
    +
+ +
+
+ +

v2022.03.01

+
+ +
    +
  • Switched to date-based release names. +
  • Added ability to have your allsky camera added to the + Allsky Map by configuring + these + settings. + Added Allsky Map Setup section to the WebUI to configure the map settings. + The Lens field now shows in the popout on the Allsky Website + (if installed). +
  • Significantly enhanced Wiki - more pages and more information on existing pages. + All known issues are described there as well as fixes / workarounds. +
  • Added an option to keograms to make them full width, even if few images were used in creating + the keogram. + In config.sh, set KEOGRAM_EXTRA_PARAMETERS="--image-expand". +
  • Added/changed/deleted settings (in config.sh unless otherwise + noted): +
      +
    • Added WEBUI_DATA_FILES - contains the name of one or more + files that contain information to + be added to the WebUI's System page. + See this Wiki + page for more information. +
    • Renamed NIGHTS_TO_KEEP to DAYS_TO_KEEP since it determines + how many days of data to keep, not just nighttime data. + If blank (""), ALL days' data are kept. +
    • Deleted AUTO_DELETE - its functionality is now in DAYS_TO_KEEP. + DAYS_TO_KEEP="" is similar to the old AUTO_DELETE=false. +
    • Added WEB_DAYS_TO_KEEP - specifies how many days of + Allsky Website images and + videos to keep, if the website is installed on your Pi. +
    • Added WEB_IMAGE_DIR in ftp-settings.sh to allow the + images to be copied to a location on your Pi (usually the Allsky Website) as well as + being copied to a remote machine. + This functionality already existed with timelapse, startrails, and keogram files. +
    +
  • The RPi camera now supports all the text overlay features as the ZWO camera, + including the Extra Text file. +
  • Removed the harmless "deprecated pixel format used" message from the timelapse log file. + That message only confused people. +
  • Improved the auto-exposure for RPi cameras. +
  • Made numerous changes to the ZWO and RPi camera's code that will make it easier to maintain and + add new features in the future. +
  • If Allsky is stopped or restarted while a file is being uploaded to a remote server, + the upload continues, eliminating cases where a temporary file would be left on the server. +
  • Decreased other cases where temporary files would be left on remote servers during uploads. + Also, uploads now perform additional error checking to help in debugging. +
  • Only one upload can now be done at a time. + Any additional uploads display a message in the log file and then exit. + This should eliminate (or signifiantly decrease) cases where a file is overwritten or not found, + resulting in an error message or a temporary file left on the server. +
  • Added a --debug option to allsky/scripts/upload.sh to aid in debugging + uploads. +
  • Upload log files are only created if there was an error; this saves writes to SD cards. +
  • The removeBadImages.sh script also only creates a log file if there was an error, + which saves one write to the SD card for every image. +
  • Allsky now stops with an error message on unrecoverable errors (e.g., no camera found). + It used to keep restarting and failing forever. +
  • More meaningful messages are displayed as images. + For example, in most cases "ERROR. See /var/log/allsky.log" messages have been + replaced + with messages containing additional information, for example, + "*** ERROR *** Allsky Stopped! ZWO camera not found!". +
  • If Allsky is restarted, a new "Allsky software is restarting" message is displayed, + instead of a "stopping" followed by "starting" message. +
  • The timelapse debug output no longer includes one line for each of several thousand images + proced. + This make it easier to see any actual errors. +
  • The Camera Settings page of the WebUI now displays the minimum, + maximum, + and default values in a popup for numerical fields. +
  • Startrails and Keogram creation no longer crash if invalid files are found. +
  • Removed the allsky/scripts/filename.sh file. +
  • The RPi Gamma value in the WebUI was renamed to Saturation, + which is what it always adjusted; Gamma was incorrect. +
  • Known issues: +
      +
    • The startrails and keogram programs don't work well if you bin differently during the + day and night. + If you don't save daytime images this won't be a problem. +
    • The minimum, maximum, and default values in the Camera + Settings page of the WebUI, + especially for the RPi camera, aren't always correct. + This is especially try if running on the Bullseye operating system, where many of the + settings changed. +
    +
+
+ + +

0.8.3

+
+ +
    +
  • Works on Bullseye operating system. +
  • RPi version: +
      +
    • Has an improved auto-exposure algorithm. + To use it, set CAPTURE_EXTRA_PARAMETERS="-daymean 0.5 -nightmean 0.2" + in config.sh. +
    • Has many new settings including support for most of the text overlay features that are + supported by the ZWO version. +
    +
  • New and changed config.sh variables, + see the Software Settings + Wiki page for more information: +
      +
    • IMG_UPLOAD_FREQUENCY specifies how often the image should + be uploaded to a website. + Useful with slow uplinks or metered Internet connections. +
    • IMG_CREATE_THUMBNAILS specifies whether or not thumbnails + should be created for each image. +
    • REMOVE_BAD_IMAGES now defaults to "true" since bad-image + detection is now done after a + picture is saved rather than once for all pictures at the end of the night. + This helps decrease problems when creating startrails, keograms, and timelapse videos. +
    • IMG_PREFIX: no longer used - the name of the image used + by the websites is now + whatever you specify in the WebUI (default: image.jpg). +
    • +
      When upgrading to 0.8.3 you MUST follow the steps listed + here. +
      +
    +
  • Replaced saveImageDay.sh and saveImageNight.sh with + saveImage.sh that has improved functionality, + including passing the sensor temperature to the dark subtraction commands, + thereby eliminating the need for the temperature.txt file. +
  • The image used by the websites (default: image.jpg) + as well as all temporary files are now written to allsky/tmp. +
    If you are using the Allsky Website you will need to change the + imageName variable in + /var/www/html/allsky/config.js to + "/current/tmp/image.jpg". +
    +
  • You can significanly reduce wear on your SD card by making allsky/tmp + a memory-based + filesystem. +
+
+ + +

0.8.1

+
+ +
    +
  • Rearranged the directory structure. +
  • Created a Wiki with additional documentation and troubleshooting tips. +
  • Renamed several variables in config.sh and ftp-settings.sh. +
  • CAMERA type of "auto" is no longer supported - you must specify + "ZWO" or "RPi". +
  • Startrails and keograms are now created using all CPUs on the Pi, drastically speeding up + creation time. +
  • Installing the WebUI now preserves any website files (keograms, startrails, etc.) you have. + This allows for non-destructive updates of the WebUI. +
  • New script called upload.sh centralizes all the upload code from other scripts, + and can be used to debug uploading issues. + See this Wiki + page for more information. +
  • The RPi camera does much better auto-exposure if you set the -mode-mean and + -autoexposure options. +
  • The WebUI will now show the Pi's throttle and low-voltage states, which is useful for debugging. +
  • Darks work better. +
  • Many bug fixes, error checks, and warnings added. +
+
+ + +

0.8

+
+ +
    +
  • Workaround for ZWO daytime autoexposure bug. +
  • Improved exposure transitions between day and night so there's not such a huge change in + brightness. +
  • Decrease in ZWO sensor temperature. +
  • Lots of new settings, including splitting some settings into day and night versions. +
  • Error checking and associated log messages added in many places to aid in debugging. +
  • Ability to have "notification" images displayed, such as "Allsky is starting up" and "Taking + dark frames". +
  • Ability to resize uploaded images. +
  • Ability to set thumbnail size. +
  • Ability to delete bad images (corrupt and too light/dark). +
  • Ability to set an image file name prefix. +
  • Ability to reset USB bus if ZWO camera isn't found (requires uhubctl command to be + installed). +
  • Ability to specify the format of the time displayed on images. +
  • Ability to have the temperature displayed in Celcius, Fahrenheit, or both. +
  • Ability to set bitrate on timelapse video. +
+
+ + +

0.7

+
+ +
    +
  • Added Raspberry Pi camera HQ support based on Rob Musquetier's fork. +
  • Added support for x86 architecture (Ubuntu, etc.). +
  • Temperature dependant dark frame library. +
  • Added browser-based script editor. +
  • Added configuration variables to crop black area around image. +
  • Added timelapse frame rate setting. +
  • Changed font size default value. +
+
+ + +

0.6

+
+ +
    +
  • Added daytime exposure and auto-exposure capability. +
  • Added -maxexposure, -autoexposure, -maxgain, + -autogain options. + Note that using autoexposure and autogain at the same time may produce unexpected results (black + frames). +
  • Autostart is now based on systemd and should work on all raspbian based systems, including + headless distributions. + Remote controlling will not start multiple instances of the software. +
  • Replaced nodisplay option with preview argument. + No preview in autostart mode. +
  • When using the WebUI, camera options can be saved without rebooting the RPi. +
  • Added a publicly accessible preview to the WebUI: public.php. +
  • Changed exposure unit to milliseconds instead of microseconds. +
+
+ + +

0.5

+
+ +
    +
  • Added Startrails (image stacking) with brightness control. +
  • Keograms and Startrails generation is now much faster thanks to a rewrite by Jarno Paananen.. +
+
+ + +

0.4

+
+ +
    +
  • Added Keograms (summary of the night in one image). +
+
+ + +

0.3

+
+ +
    +
  • Added dark frame subtraction. +
+
+ + +

0.2

+
+ +
    +
  • Separated camera settings from code logic. +
+
+ + +

0.1

+
+ +
    +
  • Initial release. +
+
+ + +
+
+ diff --git a/html/documentation/css/custom.css b/html/documentation/css/custom.css index 7720004eb..3ab00974e 100644 --- a/html/documentation/css/custom.css +++ b/html/documentation/css/custom.css @@ -1,3 +1,12 @@ +:root { + --navbar-background-color: #f8f8f8; + --navbar-border-color: #e7e7e7; + --navbar-background-color-dark: #272727; + --navbar-border-color-dark: #333333; + --WebUI-setting-color: #00009b; + --WebUI-setting-color-dark: #68a0ff; +} + .page-header { margin: 20px 0 20px; } @@ -14,12 +23,12 @@ height: 100%; } .navbar-header .navbar-brand { - padding-left: 5px; + padding-left: 5px; } .navbar-brand img { - height: 50px; - display:inline-block; - margin-right: 15px; + height: 50px; + display:inline-block; + margin-right: 15px; } .version-title { margin-left: 40px; @@ -51,7 +60,7 @@ } .right-panel .panel:first-of-type { - margin: 20px 0; + margin: 20px 0; } .panel-primary { @@ -60,7 +69,7 @@ .panel-primary>.panel-heading { border-color: #6b6b6b; - background-color: #6b6b6b; + background-color: #6b6b6b; } .panel-footer { @@ -78,119 +87,118 @@ } .switch-title { - margin-bottom: 6px; + margin-bottom: 6px; } .switch-field { - font-family: "Lucida Grande", Tahoma, Verdana, sans-serif; - margin: 3px 2px -15px 0; - display: inline-block; - overflow: hidden; + font-family: "Lucida Grande", Tahoma, Verdana, sans-serif; + margin: 3px 2px -15px 0; + display: inline-block; + overflow: hidden; } .switch-field input { - position: absolute !important; - clip: rect(0, 0, 0, 0); - height: 1px; - width: 1px; - border: 0; - overflow: hidden; + position: absolute !important; + clip: rect(0, 0, 0, 0); + height: 1px; + width: 1px; + border: 0; + overflow: hidden; } .switch-field label { - float: left; - display: inline-block; - width: 60px; - background-color: #e4e4e4; - color: rgba(0, 0, 0, 0.6); - font-size: 14px; - font-weight: normal; - text-align: center; - text-shadow: none; - padding: 6px 14px; - border: 1px solid rgba(0, 0, 0, 0.2); - -webkit-box-shadow: inset 0 1px 3px rgba(0, 0, 0, 0.3), 0 1px rgba(255, 255, 255, 0.1); - box-shadow: inset 0 1px 3px rgba(0, 0, 0, 0.3), 0 1px rgba(255, 255, 255, 0.1); + float: left; + display: inline-block; + width: 60px; + background-color: #e4e4e4; + color: rgba(0, 0, 0, 0.6); + font-size: 14px; + font-weight: normal; + text-align: center; + text-shadow: none; + padding: 6px 14px; + border: 1px solid rgba(0, 0, 0, 0.2); + -webkit-box-shadow: inset 0 1px 3px rgba(0, 0, 0, 0.3), 0 1px rgba(255, 255, 255, 0.1); + box-shadow: inset 0 1px 3px rgba(0, 0, 0, 0.3), 0 1px rgba(255, 255, 255, 0.1); } .switch-field label:hover { - cursor: pointer; + cursor: pointer; } .switch-field input:checked + label { - background-color: #3a9e55; - color: white; - font-weight: bold; - -webkit-box-shadow: none; - box-shadow: none; + background-color: #3a9e55; + color: white; + font-weight: bold; + -webkit-box-shadow: none; + box-shadow: none; } .switch-field label:first-of-type { - border-radius: 4px 0 0 4px; + border-radius: 4px 0 0 4px; } .switch-field label:last-of-type { - border-radius: 0 4px 4px 0; + border-radius: 0 4px 4px 0; } h2 { - margin-top:0; + margin-top:0; } .nav>li>span { - position: relative; - display: block; - padding: 10px 15px; - color: #4b4a4b; - cursor: pointer; + position: relative; + display: block; + padding: 10px 15px; + color: #4b4a4b; + cursor: pointer; } body.dark { - background-color: #222222; - color: #888888; + background-color: #222222; + color: #888888; } .dark #page-wrapper { - background-color: #171717; - border-left: #333333; + background-color: #171717; + border-left: #333333; } .dark .navbar-default { - background-color: #272727; - border-color: #333333; + background-color: var(--navbar-background-color-dark); + border: 1px solid var(--navbar-border-color-dark); } .dark .sidebar ul li { - border-bottom: 1px solid #333333; + border-bottom: 1px solid #333333; } .dark .sidebar ul li a, .dark .sidebar ul li span { - color: #888888; + color: #888888; } .dark .sidebar ul li a.active { - background-color: #171717; + background-color: #171717; } .dark .nav>li>a:hover { - background-color: #1D1D1D; + background-color: #1D1D1D; } .dark .panel { - background-color: #272727; + background-color: #272727; } -.dark .panel-default, -.dark .panel-primary { - border-color: #333333; +.dark .panel-default { + border-color: #333333; } .dark .panel-heading, .dark .panel-footer { - color: #888888; - border-color: #333333; - background-color: #171717; + color: #888888; + border-color: #333333; + background-color: #171717; } .alert { @@ -225,51 +233,55 @@ table .alert-dismissable { .dark .alert-message, .dark .alert-info { color: #23485a; border-color: #99ccdd; - background-color: #8b989f; + background-color: #8b989f; } .dark .alert-success { color: #95bb85; border-color: #8f9d84; - background-color: #044a05; + background-color: #044a05; } .dark .alert-warning { color: #c0bba0; border-color: #c0bba0; - background-color: #8a6b3b; + background-color: #8a6b3b; } .dark .alert-danger { color: #f1b1b1; border-color: #f1b1b1; - background-color: #a94442; + background-color: #a94442; +} +.alert-danger { + border-top-width: 10px; + border-bottom-width: 10px; } .dark .form-control, .dark input[type="text"], .dark input[type="number"], .dark select { - background-color: #171717; - color: #888888; - border-color: #555555; + background-color: #171717; + color: #888888; + border-color: #555555; } .dark .progress { - background-color: #171717; + background-color: #171717; } .dark .progress-bar { - color: #ccc; + color: #ccc; } .dark .progress-bar-success { - background-color: #276b27; + background-color: #276b27; } .dark .progress-bar-info { - background-color: #2a5f6f; + background-color: #2a5f6f; } .dark .progress-bar-warning { - background-color: #a7742d; + background-color: #a7742d; } .dark .table-striped>tbody>tr:nth-of-type(odd) { - background-color: #171717; + background-color: #171717; } .dark .table>tbody>tr>td, @@ -278,21 +290,21 @@ table .alert-dismissable { .dark .table>tfoot>tr>th, .dark .table>thead>tr>td, .dark .table>thead>tr>th { - border-color: transparent; + border-color: transparent; } .dark .navbar-default .navbar-collapse { - border-color: transparent; + border-color: transparent; } .dark .switch-field label { - background-color: #171717; - color: #888888; + background-color: #171717; + color: #888888; } .dark .switch-field input:checked + label { - background-color: #1d5f2f; - color: #171717; + background-color: #1d5f2f; + color: #171717; } /* Buttons */ @@ -305,12 +317,12 @@ table .alert-dismissable { } .dark .btn-primary { color: #bfbfbf; - border-color: #1475a7; - background-color: #1d5a79; + border-color: #1475a7; + background-color: #1d5a79; } .btn-success { - border: 2px solid #3e743e; + border: 2px solid #3e743e; } .btn-success:hover { color: black; @@ -318,53 +330,56 @@ table .alert-dismissable { } .dark .btn-success { color: #bfbfbf; - border-color: #5ea853; + border-color: #5ea853; background-color: #338833; } .btn-warning { - border: 2px solid #ac5b09; - background-color: #e48d12; + border: 2px solid #ac5b09; + background-color: #e48d12; } .btn-warning:hover { color: black; - background-color: #ffa019; + background-color: #ffa019; } .dark .btn-warning { color: #bfbfbf; - background-color: #c46d0d; - border-color: #e48d2d; + background-color: #c46d0d; + border-color: #e48d2d; } .btn-danger { color: white; - border: 2px solid #bf0b06; + border: 2px solid #bf0b06; background-color: #f13636; } -.btn-danger:hover { +.btn-danger:hover { color: black; background-color: #f96f6f; } .dark .btn-danger { color: #bfbfbf; - background-color: #8d3431; - border-color: #ad5451; + background-color: #8d3431; + border-color: #ad5451; } .btn-advanced { - border: 2px solid #c9c46a; + border: 2px solid #c9c46a; } .btn-advanced:hover { color: black; - background-color: #e3e3a3; + background-color: #e3e3a3; } .advanced, .btn-advanced { /* "advanced" options are highlighted in same color as button */ - background-color: #fffcc0; + background-color: #fffcc0; } .dark .advanced, .dark .btn-advanced { color: #bfbfbf; border-color: #bdb173; - background-color: #837a2f; + background-color: #837a2f; +} +.advanced-nocolor { + background-color: inherit; } .dark .btn-primary:hover, @@ -379,38 +394,38 @@ table .alert-dismissable { @media screen and (max-width: 795px) { /* was 767 */ - .dark .table-responsive { - border-color: transparent; - } - - .navbar { - height: 52px; - } - - .navbar-brand img { - height: 29px; - } - - .switch-field { - display: block; - /* height: 40px; /* was 50 */ - } + .dark .table-responsive { + border-color: transparent; + } + + .navbar { + height: 52px; + } + + .navbar-brand img { + height: 29px; + } + + .switch-field { + display: block; + /* height: 40px; /* was 50 */ + } - form thead { - display: none; - } + form thead { + display: none; + } } -/* Headers in camera options */ +/* Headers in Allsky settings */ .settingsHeader { - background-color: #aaa; - text-align: center; - font-weight: bold; - font-size: 125%; - padding: 3px 0 3px 0; + background-color: #aaa; + text-align: center; + font-weight: bold; + font-size: 1.25em; + padding: 3px 0 3px 0; } .dark .settingsHeader { - background-color: #555; + background-color: #555; } .boxShadow { @@ -527,11 +542,11 @@ select { } .WebUISetting { /* Matches what's in the WebUI */ font-family: "Helvetica Neue",Helvetica,Arial,sans-serif; - color:#00009b; + color: var(--WebUI-setting-color); font-weight: bold; } .dark .WebUISetting { - color: #68a0ff; + color: var(--WebUI-setting-color-dark); } .WebUIValue { padding: 0 3px 0 3px; @@ -561,9 +576,42 @@ select { } .cursorPointer { - cursor: pointer; + cursor: pointer; } .live_container { background-color: black; margin-bottom: 15px; } + +div.sticky { + position: -webkit-sticky; + position: sticky; + top: 0; + margin-top: 10px; + padding: 10px; + background-color: var(--navbar-background-color); + border: 2px solid var(--WebUI-setting-color); +} +.dark div.sticky { + background-color: var(--navbar-background-color-dark); + border-color: var(--WebUI-setting-color-dark); +} +#backToTopBtn { + display: none; /* Hidden by default */ + position: fixed; /* Fixed/sticky position */ + bottom: 20px; /* Place the button at the bottom of the page */ + right: 30px; /* Place the button 30px from the right */ + z-index: 99; /* Make sure it does not overlap */ + border: none; /* Remove borders */ + outline: none; /* Remove outline */ + background-color: #555; /* Set a background color */ + color: white; /* Text color */ + cursor: pointer; /* Add a mouse pointer on hover */ + padding: 15px; /* Some padding */ + border-radius: 10px; /* Rounded corners */ + font-size: 18px; /* Increase font size */ +} + +#backToTopBtn:hover { + background-color: #888; /* Add a lighter-grey background on hover */ +} diff --git a/html/documentation/css/documentation.css b/html/documentation/css/documentation.css index a4f8c50c6..b2a4d8f59 100644 --- a/html/documentation/css/documentation.css +++ b/html/documentation/css/documentation.css @@ -898,6 +898,16 @@ b, strong { padding: 3px 6px; font-size: 12px; } +.btn-not-real { + cursor: auto; + pointer-events: none; +} +.btn-advanced { + border: 2px solid #c9c46a; +} +.advanced, .btn-advanced { /* "advanced" options are highlighted in same color as button */ + background-color: #fffcc0; +} .markdown-body .systemPageAdditionsLineType { font-weight: bold; diff --git a/html/documentation/explanations/SSL.html b/html/documentation/explanations/SSL.html index bc64bb5f1..d793d8c7d 100644 --- a/html/documentation/explanations/SSL.html +++ b/html/documentation/explanations/SSL.html @@ -30,33 +30,193 @@ This is desirable (really, mandatory) when accessing the Pi from the Internet. If your Pi is only available on a local network, SSL isn't needed.

+ +

Requirements

+
    +
  1. You must have Allsky working on your Pi prior to installing SSL. +
  2. You must already have an Internet domain name for your Pi. + The instructions below use myallsky.com but + replace that with your domain name. +
+ + +
+Installing SSL requires that your Pi be accessible via the Internet and +can be a huge security risk if not done correctly. +

+Do not attempt unless you know what you are doing. +The Allsky developers are not responsible for any loss or damages. +

+There are additional steps you should take to secure your Pi that aren't +described in these instructions. +
+If you aren't familiar with the following you're probably not qualified to install SSL: +
    +
  • installing a firewall and a VLAN +
  • obtaining an Internet domain name +
+
+ +

+There are many ways to implement SSL; +the instructions below use +Let's Encrypt SSL +which is popular and free software package. +

+ + +

Software installation

+Install some software on your Pi +(you can copy/paste these lines into a terminal window): +
+apt-get update
+apt-get install software-properties-common
+add-apt-repository ppa:certbot/certbot
+apt-get update
+apt-get install certbot
+mkdir ~/allsky/config/ssl         
+chmod 775 ~/allsky/config/ssl
+
+certbot certonly --webroot -w /config/ssl/myallsky.com -d myallsky.com
+
+ +

+The cerbot command will prompt for your email and a couple other things. +It will then create the SSL certificate and private key. +Take note of the IMPORTANT NOTES that are displayed. +

-Allsky on the Pi uses the lighttpd web server, -so it needs to be configured to use SSL. -This page -describes how to configure and use Let's Encrypt SSL with lighttpd. +Combine the certificate and private key into one file: +

+sudo chmod 750 /etc/letsencrypt/live
+cd /etc/letsencrypt/live/myallsky.com
+cat cert.pem privkey.pem | sudo tee web.pem
+

-
-You may need to to change the permissions of /etc/letsencrypt/live -in order to be able to read and write and complete the concatenation step. -Execute sudo chmod 640 /etc/letsencrypt/live. + +

Web server configuration

+Configure the web server (called lighttpd to use SSL and redirects any +"http" requests to your Pi with "https" requests. +

+In your favorite text editor, +create a file called 98-myallsky.com.conf +containing the following lines: +

+server.modules += ( "mod_openssl" )
+
+$SERVER["socket"] == ":443" {
+	ssl.engine = "enable"
+	ssl.pemfile = "/etc/letsencrypt/live/myallsky.com/web.pem"	# Combined Certificate
+	ssl.ca-file = "/etc/letsencrypt/live/myallsky.com/chain.pem"	# Root CA
+	server.name = "myallsky.com"
+
+	# Uncomment the next line if you want the web server to log access requests.
+	# accesslog.filename = "/var/log/lighttpd/myallsky.com_access.log"
+}
+
+$HTTP["scheme"] == "http" {
+	$HTTP["host"] == "myallsky.com" {
+		url.redirect = ("/.*" => "https://myallsky.com$0")
+	}
+}
+
+

+ +

+Install the file so the web server sees it: +

+sudo install -m 0644  98-myallsky.com.conf  /etc/lighttpd/conf-available
+cd /etc/lighttpd/conf-enabled
+sudo ln -s  "../conf-available/98-myallsky.com.conf"  .
+sudo systemctl restart lighttpd
+
+

+ + +

Test

+Try to access your Pi via +http://myallsky.com +and +https://myallsky.com. +You should see the WebUI in both cases. +

+ + +

Post-installation steps

+Let's Encrypt certificates are only valid for 90 days. +You need to configure the Pi to renew your certificates before they expire +(or do it manually). + +

+First, run this command to simulate automatic renewal of your certificate: +

+certbot renew --dry-run
+
+

+You should see output similar to this: +

+Saving debug log to /var/log/letsencrypt/letsencrypt.log
+-------------------------------------------------------------------------------
+Processing /etc/letsencrypt/renewal/myallsky.com.conf
+-------------------------------------------------------------------------------
+Cert not due for renewal, but simulating renewal for dry run
+Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
+Renewing an existing certificate
+Performing the following challenges:
+http-01 challenge for myallsky.com
+Waiting for verification...
+Cleaning up challenges
+-------------------------------------------------------------------------------
+new certificate deployed without reload, fullchain is
+/etc/letsencrypt/live/myallsky.com/fullchain.pem
+-------------------------------------------------------------------------------
+** DRY RUN: simulating 'certbot renew' close to cert expiry
+**          (The test certificates below have not been saved.)
+Congratulations, all renewals succeeded. The following certs have been renewed:
+  /etc/letsencrypt/live/myallsky.com/fullchain.pem (success)
+** DRY RUN: simulating 'certbot renew' close to cert expiry
+**          (The test certificates above have not been saved.)
+
+

+ +

+Prior the the certificates expiring, run: +

+certbot renew
+cd /etc/letsencrypt/live/myallsky.com
+cat cert.pem privkey.pem | sudo tee web.pem
+
+
+You can renew your Let's Encrypt certificates at most +30 days before they expire - two weeks prior to expiration is a good goal.
+

-Notes: + +

Notes

    -
  • The Pi runs a version of the Debian operating system. -
  • The lighttpd web server is already installed and running so you can skip - the step to install it. -
  • The "document root" of the webserver is ~/allsky/html. -
  • When you are done, make a copy of the /etc/letsencrypt - directory and store on something other than your Pi in case your Pi crashes. +
  • Your SSL-related files are stored in + ~/allsky/config/ssl and will be + preserved across Allsky upgrades. +
  • When you are done installing and testing SSL, make a copy of the following + and store on something other than your Pi in case your Pi crashes: +
      +
    • /etc/letsencrypt directory +
    • ~/allsky/config/ssl directory +
    • /etc/lighttpd/conf-available/98-myallsky.com.conf +
+

Source of instructions

+These instruction were obtain from +https://www.itzgeek.com +and modified for Allsky use. + + - diff --git a/html/documentation/explanations/SystemPageAdditions.html b/html/documentation/explanations/SystemPageAdditions.html index 024948bcf..8c8e45055 100644 --- a/html/documentation/explanations/SystemPageAdditions.html +++ b/html/documentation/explanations/SystemPageAdditions.html @@ -395,7 +395,7 @@

Determine what the buttons should look like

You like green so will make the dew heater toggle button green by specifying its button color as green.

You want an icon on the button since the other buttons have icons, -so you go to Font Awesome +so you go to Font Awesome to look for an icon (Allsky uses Font Awesome version 6). You pick the "random" icon so set the button's FA icon field to random.

diff --git a/html/documentation/explanations/darkFrames.html b/html/documentation/explanations/darkFrames.html index 7038cbea7..de225827f 100644 --- a/html/documentation/explanations/darkFrames.html +++ b/html/documentation/explanations/darkFrames.html @@ -106,8 +106,7 @@

How do I take and use darks?

  • In the WebUI open the Camera Settings page and set Take Dark Frames to Yes.
  • Click on the - - button. + Save changes button. This restarts Allsky with the new settings.
  • Dark frames are created in the ~/allsky/darks directory. A new dark is created every time the sensor temperature changes by 1 degree C. @@ -127,8 +126,7 @@

    How do I take and use darks?

  • Use Dark Frames to Yes
  • Click on the - - button. + Save changes button. This will restart Allsky taking light frames, and subtract dark frames at night.
  • You will get an error message if there aren't any dark frames. diff --git a/html/documentation/img/saveChangesButton.png b/html/documentation/img/saveChangesButton.png deleted file mode 100644 index 0479e7fc5..000000000 Binary files a/html/documentation/img/saveChangesButton.png and /dev/null differ diff --git a/html/documentation/img/showAdvancedOptionsButton.png b/html/documentation/img/showAdvancedOptionsButton.png deleted file mode 100644 index 68921b627..000000000 Binary files a/html/documentation/img/showAdvancedOptionsButton.png and /dev/null differ diff --git a/html/documentation/installations/Allsky.html b/html/documentation/installations/Allsky.html index c5801f57c..d944ef2a4 100644 --- a/html/documentation/installations/Allsky.html +++ b/html/documentation/installations/Allsky.html @@ -229,12 +229,11 @@

    Post installation

  • Go to the Allsky Settings page.
  • Optionally click the - + Show advanced options button. button to show the advanced options.
  • Make any necessary changes.
  • Click on the - - button. + Save changes button. Allsky will (re)start.
  • If instructed to update the config.sh and/or the ftp-settings.sh files: diff --git a/html/documentation/installations/AllskyWebsite.html b/html/documentation/installations/AllskyWebsite.html index cc38e7c89..ffb548d38 100644 --- a/html/documentation/installations/AllskyWebsite.html +++ b/html/documentation/installations/AllskyWebsite.html @@ -16,6 +16,7 @@ + Installing and upgrading Allsky Website @@ -41,7 +42,7 @@

    Install on a Pi

    If you have an existing Allsky Website on your Pi, see the section below, otherwise -skip to the Installing the Website section. +skip to the Installing the Website section below.

    Steps for existing Websites

    @@ -49,14 +50,16 @@

    Steps for existing Websites

    The location of the prior website depends upon its version. In both cases the directory name is allsky. The parent directory is: -
      -
    • /var/www/html for Website version v2022.03.01+ or before. -
    • ~/allsky/html for newer versions. +
        +
      • /var/www/html + for Website version v2022.03.01+ or before. +
      • ~/allsky/html + for version v2023.05.01 or later.

      -Perform one of the steps below, +Perform one of the steps below, then continue to the Installing the Website section:

      1. Upgrade the old version. @@ -104,6 +107,8 @@

        Installing the Website

      2. Update the ftp-settings.sh file via the WebUI's Editor page so Allsky knows where to copy the files to. + See the instructions for + updating the ftp-settings.sh file.

    @@ -119,7 +124,7 @@

    Install on a remote server

    machine that is accessible on the Internet.
    If you want to make your Pi available on the Internet, see -these instructions. +these instructions.
    Be careful if you do this - if done incorrectly your Pi may be insecure.
    @@ -143,13 +148,20 @@

    Install on a remote server

    allsky directory on the remote server.
  • Do not configure the Website yet - you'll do that below.
  • On the Pi: -
      +
      1. Update the config.sh file via the WebUI's Editor page so Allsky knows which files to upload (e.g., startrails, timelapse, etc.).
      2. Update the ftp-settings.sh file via the WebUI's Editor page so Allsky knows where on the remote server to upload the files to. + See the instructions for + updating the ftp-settings.sh file. +
        + This step MUST be done before the next step since the next step + tries to upload a test file to your remote server and it will fail + if you haven't configured the settings properly. +
      3. Run cd ~/allsky; website/install.sh --remote. This will upload a default configuration file to your server, leaving the master copy on the Pi. @@ -181,7 +193,7 @@

        Install on a remote server

        do so via WebUI (follow the first two steps above). Do NOT edit the configuration file directly on the remote server.
  • - +
  • Your remote server is now ready.
  • Give your family and friends the URL to your Allsky Website so they can enjoy your images! @@ -220,13 +232,14 @@

    Post installation - local and remote

    Changes to both local Websites and remote Websites are both done via the WebUI.

    -
    +

    If your prior version of the Allsky Website was in /var/www/html/allsky note that there is only one configuration file in the new Website (configuration.json) that replaces the two older files. -

    +

    +
    If you have both a local and a remote Website, they each have their own configuration file which is diff --git a/html/documentation/miscellaneous/AllskyMap.html b/html/documentation/miscellaneous/AllskyMap.html index de9b7b8c5..25a541086 100644 --- a/html/documentation/miscellaneous/AllskyMap.html +++ b/html/documentation/miscellaneous/AllskyMap.html @@ -29,7 +29,7 @@

    Overview

    Starting with version 0.8.3.3 of Allsky, you can automatically have your allsky camera(s) added to the global -Allsky Map . +Allsky Map. This map shows the location of all known allsky cameras as well as basic information about them. Click on the image below to go to the Allsky Map.

    @@ -60,7 +60,7 @@

    Overview

    See the "Allsky Settings" Page section of the -Allsky Settings +Allsky Settings page for a description of each map-related setting in the WebUI.

    diff --git a/html/documentation/miscellaneous/pickingHardware.html b/html/documentation/miscellaneous/pickingHardware.html index 479f3c234..433b805e4 100644 --- a/html/documentation/miscellaneous/pickingHardware.html +++ b/html/documentation/miscellaneous/pickingHardware.html @@ -51,9 +51,9 @@

    Raspberry Pi choices

    Pi Zero 2 may not work in the future.

    -We have not found any other brand devices to be compatible with the Pi, -despite what their marketing information says. -We cannot support non-Raspberry Pi devices. +We have not found any Pi clones except Le Potato to be compatible with Allsky. +The Le Potato clone works with a ZWO ASI120 camera; +no other cameras, including any RPi cameras, were tested, so buyer beware.
    diff --git a/html/documentation/modules/dew-heater.png b/html/documentation/modules/dew-heater.png index a221643c4..9fa0af9c9 100755 Binary files a/html/documentation/modules/dew-heater.png and b/html/documentation/modules/dew-heater.png differ diff --git a/html/documentation/overlays/overlays.html b/html/documentation/overlays/overlays.html index 34be697ad..e8f11f6ea 100755 --- a/html/documentation/overlays/overlays.html +++ b/html/documentation/overlays/overlays.html @@ -70,7 +70,8 @@

    Features

    to position them.
  • Customisable Interface - The Overlay Editor - user interface can be highly customised.
  • + user interface can be highly customised. +
  • Font Manager - You can upload any TrueType font and use it in the overlays or use any font already on your Pi.
  • Variable Manager - Provides a library of fields that @@ -181,38 +182,38 @@

    Fields and Variables

    Variable names: -

      -
    • Must begin with a letter a - z or A - Z. -
    • The remaining characters may contain any number of letters, number 0 - 9, - or the underscore character _. -
    • Should be prefixed with a string that's unique to you like your initials, - to avoid conflicting with the names of Allsky system variables. - This prefix can be anything except AS_ and ALLSKY_. -
    • Should make sense to you. - You are more likely to remember what ${MY_AMBIENT_TEMP} - means than ${MY_AT}. -
    • By convention are UPPERCASE, but you can mix case, - however, variable names are case-sensitive, - so ${MY_ABIENT_TEMP} and ${MY_ABIENT_temp} - are different variables. -
    +
      +
    • Must begin with a letter a - z or A - Z. +
    • The remaining characters may contain any number of letters, number 0 - 9, + or the underscore character _. +
    • Should be prefixed with a string that's unique to you like your initials, + to avoid conflicting with the names of Allsky system variables. + This prefix can be anything except AS_ and ALLSKY_. +
    • Should make sense to you. + You are more likely to remember what ${MY_AMBIENT_TEMP} + means than ${MY_AT}. +
    • By convention are UPPERCASE, but you can mix case, + however, variable names are case-sensitive, + so ${MY_ABIENT_TEMP} and ${MY_ABIENT_temp} + are different variables. +

    A variable is a variable name enclosed within ${}, for example, ${sEXPOSURE}.

    -

    - If you see a variable displayed on the overlay as ??? it usually means - the variable is undefined - either it's not in the - Variable Manager at all (could be a typo), - or is only in the All Variables tab. - The variable needs to be defined and copied to the - Allsky Variables tab. -
    Undefined variables will have a line in - /var/log/allsky.log like: - ERROR: ${T2} has no variable type; check 'fields.json'. -

    +

    + If you see a variable displayed on the overlay as ??? it usually means + the variable is undefined - either it's not in the + Variable Manager at all (could be a typo), + or is only in the All Variables tab. + The variable needs to be defined and copied to the + Allsky Variables tab. +
    Undefined variables will have a line in + /var/log/allsky.log like: + ERROR: ${T2} has no variable type; check 'fields.json'. +

    If a variable is displayed as ?? it usually means the variable's formatting is incorrect, for example, you tried to display a number as a date. @@ -222,7 +223,7 @@

    Fields and Variables

    /var/log/allsky.log like: ERROR: Cannot use format '%a' on Number variables like ${GAIN}.. -
    +

    Some example text fields are:

    @@ -240,7 +241,8 @@

    Fields and Variables

    24/10/2023 Displays the date from the DATE system variable. The date can be formatted in a variety of ways. -
    This field contains only a variable. +
    This field contains only a variable. + Date: ${DATE} @@ -248,12 +250,14 @@

    Fields and Variables

    Displays the text "Date: " then the date the image was taken from the DATE system variable. As above, the date can be formatted in a variety of ways. -
    This field contains text and a variable. +
    This field contains text and a variable. + Date: ${DATE} ${TIME} Date: 24/10/2023 23:12:34 - Displays the date and time the image was taken from the DATE and TIME system variables, + Displays the date and time the image was taken from the DATE and TIME system + variables, respectively. Both variables can be formatted in a variety of ways.
    This field contains text and two variables. @@ -265,7 +269,8 @@

    Fields and Variables

    Date: DATE TIME Because this field doesn't have any variables, it simply displays "Date: DATE TIME". -
    This field contains only text. +
    This field contains only text. + @@ -393,9 +398,11 @@

    2. Module variables

    For this reason it's important that the Overlay Module runs as late as possible within the module flow. As an example, the Star Count Module creates a - variable called ${STARCOUNT} and passes it to the next module.

    + variable called ${STARCOUNT} and passes it to the next module. +

    Please refer to the - documentation on each module + documentation on + each module for the variables they makes available.

    @@ -407,7 +414,8 @@

    3. Extra Data variables - advanced topic

    This is an advanced topic that requires an understanding of the Linux file system and how to manage files within it. - The Linux Basics + The Linux + Basics page should provide the understanding you need.

    As an example, assume you want to add weather data to your images. @@ -430,7 +438,8 @@

    3. Extra Data variables - advanced topic

    Dates

    When adding dates to any extra data files please ensure that the date is formatted in the same format as the - Time Format setting in WebUI

    + Time Format setting in WebUI +

    Text Files

    Text files must end with a .txt extension. @@ -463,8 +472,9 @@

    Text Files

    to ignore "extra" files when they are over a certain age. For .txt files there is a single value which is specified in the Overlay Editor settings dialog box. - See the next section for how to specify the expiration time for - .json files.

    + See the next section for how to specify the expiration time for + .json files. +

    JSON Files

    @@ -571,7 +581,7 @@

    JSON Files

    Overlay Editor default expiry setting for the other two. This works well if you only have one "extra" file. -
  • +
  • If you have multiple "extra" files it's probably best to specify an expiry time for ALL fields in all files, rather than using the @@ -580,7 +590,7 @@

    JSON Files

    If a field shouldn't have an expiry time, use "expires": 0, which disables its expiry time. -
  • + @@ -638,7 +648,7 @@

    JSON Files

    The Overlay Editor User Interface

    - The Overlay Editor web page consists of two key areas: + The Overlay Editor web page consists of two key areas:

    1. The toolbar contains icons that perform actions or @@ -654,7 +664,7 @@

      The Overlay Editor User Interface


      If you added a field by mistake, delete it by clicking on the icon (4) - before doing anything else. + before doing anything else.

      Here is a typical top portion of the editor, @@ -849,9 +859,9 @@

      The Overlay Editor User Interface

      smaller than those captured by your camera so you may not be using all of the available screen area.

      Ideally you will create overlays after you've set any - resize and crop options and the captured images size is its - "final" size.

      - + resize and crop options and the captured images size is its + "final" size.

      + @@ -867,176 +877,181 @@

      Using The Overlay Editor

      -

      The Variable Manager  

      -

      The Variable Manager is used to store - variables and text that can be used in fields. -

      -
      - +

      The Variable Manager   +

      +

      The Variable Manager is used to store + variables and text that can be used in fields. +

      +
      + -

      The Variable Manager provides a quick method - for storing details of the available variables and adding them to the overlay. - The manager comes preinstalled with a set of system variables that - can be modified but not deleted. - You can also add your own variables to the manager; typically - these will come from modules or "extra" data files.

      -

      The manager consists of two tabs: -

        -
      1. The Allsky Variables tab contains - the variables that have been defined and hence can be - added to the overlay. -
      2. The All Variables tab contains all variables - including ones that have not been defined and hence cannot be - added to the overlay. -
      -

      +

      The Variable Manager provides a quick method + for storing details of the available variables and adding them to the overlay. + The manager comes preinstalled with a set of system variables that + can be modified but not deleted. + You can also add your own variables to the manager; typically + these will come from modules or "extra" data files.

      +

      The manager consists of two tabs: +

        +
      1. The Allsky Variables tab contains + the variables that have been defined and hence can be + added to the overlay. +
      2. The All Variables tab contains all variables + including ones that have not been defined and hence cannot be + added to the overlay. +
      +

      -

      Allsky Variables tab

      - - - - - - - - - -
      - Variable Manager -
      -
        -
      • The - icon - adds the associated variable in a new text field to the overlay. - The Text Properties dialog box - will also be displayed and populated with the relevant - data for the variable.
      • -
      • The - icon - edits the variable. If the variable is a system variable then only - certain fields can be changed.
      • -
      • The - icon - deletes a variable. - You can only delete variables that you added; - there are none shown in the screenshot above. -
      • -
      +

      Allsky Variables tab

      + + + + + + + + + +
      + Variable Manager +
      +
        +
      • The + icon + adds the associated variable in a new text field to the overlay. + The Text Properties dialog box + will also be displayed and populated with the relevant + data for the variable.
      • +
      • The + icon + edits the variable. If the variable is a system variable then only + certain fields can be changed.
      • +
      • The + icon + deletes a variable. + You can only delete variables that you added; + there are none shown in the screenshot above. +
      • +
      + +

      + The Add Variable + brings up the "Add Variable" dialog box that allows you to + add your own variable and set the variable's Type and other attributes. +

      + This button does NOT add the variable to the overlay; + instead, it just adds it to the end of the + Allsky Variables list so it can be added to the overlay. +
      +

      +

      The search option in the top right can be used + to quickly search for a variable by name. + Searches are not case sensitive.

      +
      -

      - The Add Variable - brings up the "Add Variable" dialog box that allows you to - add your own variable and set the variable's Type and other attributes. -

      - This button does NOT add the variable to the overlay; - instead, it just adds it to the end of the - Allsky Variables list so it can be added to the overlay. -
      -

      -

      The search option in the top right can be used - to quickly search for a variable by name. - Searches are not case sensitive.

      -
      +
      +

      All Variables tab

      + + + + + + + + + +
      + Variable Manager All +
      +

      This tab lists every variable that Allsky can see and the last value it had, + if any. + These variables only have a Name as well as possibly their last value. + They are not yet defined, i.e., they have no attributes like + data Type, Description, Format, or Sample Data. +

      +
        +
      • Click the + icon + to define the variable's attributes + and then add the variable to the Allsky Variables tab + making it available for use on overlays. + Note that doing this does not put the variable on the overlay - + it just makes it available to be put on the overlay. +
      • +
        If this button is disabled then the variable is already + in the Allsky Variables tab.
        +
      + +

      + The Add Variable + button does the same thing as the same button in the + Allsky Variables tab. +

      +
      -

      All Variables tab

      - - - - - - - + + +
      - Variable Manager All -
      -

      This tab lists every variable that Allsky can see and the last value it had, - if any. - These variables only have a Name as well as possibly their last value. - They are not yet defined, i.e., they have no attributes like - data Type, Description, Format, or Sample Data. -

      -
        -
      • Click the - icon - to define the variable's attributes - and then add the variable to the Allsky Variables tab - making it available for use on overlays. - Note that doing this does not put the variable on the overlay - - it just makes it available to be put on the overlay. +
        +

        The "Add Variable" and "Edit Variable" dialog boxes

        +

        Clicking on the + Add Variable button + or the + + + icon above brings up a dialog box to add or edit a variable. + It's the same dialog box in either case, just with a different title. + Editing a System Variable will display a message as shown in the image below. +

        + + + + + + + - - -
        + Edit variable +
        +
          +
        • Variable Name - The name of the variable + enclosed within ${}. + For system variables the name cannot be changed. + See the Variable names + section for details on valid variable names. +
        • +
        • Description - A description of the variable. + This is only for reference and is displayed in the + Variable Manager. +
        • +
        • Format - The optional + formatting used to + display the variable. +
        • +
        • Sample Data - Sample data that is used when + displaying + sample data in the Overlay Editor. +
        • +
        • Type - Each variable has a type that + is used to determine what formatting to use. The variable types are: +
            +
          • Date - A variable that holds a date
          • +
          • Time - A variable that holds a time
          • +
          • Number - A number with, or without a decimal point
          • -
            If this button is disabled then the variable is already - in the Allsky Variables tab.
            -
          - -

          - The Add Variable - button does the same thing as the same button in the - Allsky Variables tab.

          -
        - -

        The "Add Variable" and "Edit Variable" dialog boxes

        -

        Clicking on the - Add Variable button - or the - - - icon above brings up a dialog box to add or edit a variable. - It's the same dialog box in either case, just with a different title. - Editing a System Variable will display a message as shown in the image below. -

        - - - - - - - - - -
        - Edit variable -
        -
          -
        • Variable Name - The name of the variable - enclosed within ${}. - For system variables the name cannot be changed. - See the Variable names - section for details on valid variable names. +
        • Text - Any string of characters
        • +
        • Bool - A variable that holds a boolean value - + on/off, yes/no, true/false.
        • -
        • Description - A description of the variable. - This is only for reference and is displayed in the - Variable Manager. -
        • -
        • Format - The optional - formatting used to - display the variable. -
        • -
        • Sample Data - Sample data that is used when - displaying - sample data in the Overlay Editor. -
        • -
        • Type - Each variable has a type that - is used to determine what formatting to use. The variable types are: -
            -
          • Date - A variable that holds a date
          • -
          • Time - A variable that holds a time
          • -
          • Number - A number with, or without a decimal point
          • -
          • Text - Any string of characters
          • -
          • Bool - A variable that holds a boolean value - - on/off, yes/no, true/false. -
          • -
          -
        • -
        -
        +
      + + +
      -
      -
      +
      +

      The Font Manager  

      @@ -1048,8 +1063,7 @@

      The Font Manager   - Font Manager + Font Manager @@ -1063,7 +1077,8 @@

      The Font Manager  
    2. 2 Delete Button - Deletes a font.
      It is not possible to delete a System Font and they - will not have a delete icon as shown for the first several fonts above. + will not have a delete icon as shown for the first several fonts + above.
      If a font is in use when deleted then any fields using the font will revert to the default font as @@ -1089,14 +1104,16 @@

      The Font Manager  

      To use a font it must first be installed in the Font Manager - and there are two ways to do this:

      + and there are two ways to do this: +

      1. Add font from daFont.com
        In a separate browser window, navigate to the font page on http://daFont.com that you wish to install, for example - https://www.dafont.com/led-sled.font. + https://www.dafont.com/led-sled.font. Copy the font URL to the clipboard. Click the Add Font button and enter the font URL. @@ -1111,6 +1128,12 @@

        The Font Manager   Click the Upload Font button and browse to the zip file. The fonts will be extracted from the zip file and installed.

        +

        NOTE: If you receive an error when uploading a font please check the php max + file upload size. By default this is 2Mb which may be too small for some font zip + files

        +

        The php configuration file is normally /etc/php/7.4/cgi/php.ini. Edit this file and locate the line containing 'upload_max_filesize =' and change the value. So if the exiting line says 'upload_max_filesize = 2MB' change it to 'upload_max_filesize = 20MB'

        +

        Following this change either restart lighttpd with

        sudo systemctl restart lighttpd

        +


      @@ -1127,8 +1150,7 @@

      The Image Manager   - Image Manager + Image Manager @@ -1197,7 +1219,7 @@

      Adding fields to the overlay

      The field will start off with just the variable you selected; you can add text or other variables to the field later. See the Variable Manager section for information on - the All Variables tab in the + the All Variables tab in the Variable Manager.

    3. Add a pre-defined image in a new field @@ -1223,8 +1245,8 @@

      Text Properties Editor

      - Text Properties Editor + Text Properties Editor @@ -1248,7 +1270,7 @@

      Text Properties Editor

      multiple Sample values by separating them by commas or enclosing each Sample in { }. See the screenshot above for an example of separating by commas. -
    4. +

    5. Empty Value - By default, if ALL the variables in a field have no value then the the field is not displayed. However, if this property has something in it then the whole field @@ -1261,9 +1283,10 @@

      Text Properties Editor

      If a field doesn't appear on the overlay and you're not sure why, look in /var/log/allsky.log for messages like - Adding text field T1: ${T1} failed no variable data available + Adding text field T1: ${T1} failed no variable data available -
    6. +
    7. X - The 'x' position of the field in pixels from the left of the image. If setting manually you need to take into account the @@ -1306,8 +1329,9 @@

      Image Properties Editor

      - Image Properties Editor + Image Properties Editor + @@ -1338,7 +1362,7 @@

      Image Properties Editor

      from 0 (image hidden) to 10 times the native resolution. Images that are scaled a lot can look jagged and you may want to consider getting a higher-resolution version. -
    8. + @@ -1357,8 +1381,8 @@

      Formatting Variables

      - Properties Help + Properties Help @@ -1369,12 +1393,15 @@

      Formatting Variables

      Text Properties dialog box to view the window below. After you have set a format click the "Display Sample Data" icon - on the toolbbar to show the formatted result.

      + on the toolbbar to show the formatted result. +

      Whilst the Overlay Editor is displaying sample data, - changing the Format field will update the sample.
      + changing the Format field will update the sample. +

      The number formats are based upon the formating options + href="https://learnpython.com/blog/python-string-formatting/">formating + options available in python.

      @@ -1385,8 +1412,8 @@

      Formatting Variables

      - Formats Help + Formats Help @@ -1397,21 +1424,26 @@

      Formatting Variables

      dialog then the formats window will automatically be hidden.

      • 1 - - Selecting an entry will display the available formats for that variable type.
      • + - Selecting an entry will display the available formats for that + variable type.
      • 2 - These buttons are used to amend the variable's format:
        • - -   + +   This button replaces the format on the current field with the selected format. - This is most useful when working with number formats.
        • + This is most useful when working with number + formats.
        • - -   - This button appends the selected format to the format field. - This is most useful when working with date formats.
        • + +   + This button appends the selected format to the format + field. + This is most useful when working with date + formats.
      @@ -1423,7 +1455,8 @@

      Formatting Variables

      -

      Bool formats

      +
      +

      Bool formats

      @@ -1457,7 +1490,8 @@

      Formatting Variables

      Bool formats must be lowercase.
      -

      Date and Time formats

      +
      +

      Date and Time formats

      @@ -1615,7 +1649,8 @@

      Formatting Variables

      -

      Number formats

      +
      +

      Number formats

      @@ -1758,15 +1793,18 @@

      Editor Settings tab

      snap to if dropped.
    9. Add List Page Size - The number of variables to show per page in the - Variable Manager.
    10. + Variable Manager. +
    11. Add Field Brightness - When adding a new field all other fields will be set to this brightness to make the new field easier to see. - Values range from 0 (darkest) to 100 (each field's full brightness).
    12. + Values range from 0 (darkest) to 100 (each field's full brightness). +
    13. Select Field Brightness - When selecting a field all other fields will be set to this opacity to make the new field easier to see. - Values range from 0 (darkest) to 100 (each field's full brightness).
    14. + Values range from 0 (darkest) to 100 (each field's full brightness). +
    15. Zoom with Mouse Wheel - When enabled, scrolling the mouse wheel will zoom the overlay.
    16. Background Image Brightness - Brightness of the @@ -1782,7 +1820,8 @@

      Editor Settings tab

    17. -

      When changing the brightness settings you may need to click somewhere in the +
      When changing the brightness settings you may need to click + somewhere in the overlay in order for the change to take affect.

      @@ -1816,7 +1855,7 @@

      Sun / Moon / Planets / Satellites

      even if their associated Include checkboxes are NOT selected. Their values will not appear in the overlay until the checkboxes are enabled. -
      @@ -1882,7 +1921,8 @@

      The Sun

      This format can be overriden in the Date Time format field of the Overlay Module settings in the Module Manager. -
      See the screenshot below for this field.

      +
      See the screenshot below for this field. +

      @@ -1891,7 +1931,8 @@

      The Sun

      -

      The Moon

      +
      +

      The Moon

      The following Moon-related variables are available in the Variable Manager:

      @@ -1929,7 +1970,8 @@

      The Sun

      -

      The planets

      +
      +

      The planets

      The following planet-related variables are available in the Variable Manager:

      @@ -2074,7 +2116,8 @@

      The Sun

      -

      Satellites (including the ISS and Hubble)

      +
      +

      Satellites (including the ISS and Hubble)

      The Overlay Module can generate position elements for the International Space Station (ISS), the Hubble Space Telescope, @@ -2082,7 +2125,8 @@

      The Sun

      Overlay Editor's Norad IDs setting. IDs must be comma-separated.

      -

      To find NORAD IDs browse to the Celestrak +

      To find NORAD IDs browse to the Celestrak website and search for the desired objects.

      @@ -2158,4 +2202,4 @@

      The Sun

      - + \ No newline at end of file diff --git a/html/documentation/settings/AllskySettingsPage.png b/html/documentation/settings/AllskySettingsPage.png index 4fb04b5f5..f39822667 100644 Binary files a/html/documentation/settings/AllskySettingsPage.png and b/html/documentation/settings/AllskySettingsPage.png differ diff --git a/html/documentation/settings/EditorPage.png b/html/documentation/settings/EditorPage.png index 922e2e933..1c7fcb622 100644 Binary files a/html/documentation/settings/EditorPage.png and b/html/documentation/settings/EditorPage.png differ diff --git a/html/documentation/settings/allsky.html b/html/documentation/settings/allsky.html index 401c38c20..fe2d206ea 100644 --- a/html/documentation/settings/allsky.html +++ b/html/documentation/settings/allsky.html @@ -29,7 +29,7 @@ These settings must be changed via the WebUI rather than manually editing files since the WebUI performs error checking and updates other files as appropriate.

      -There WebUI contains many pages; the two used to changes settings are +The WebUI contains many pages; the two used to changes settings are the Allsky Settings and Editor pages, and are described below. @@ -42,8 +42,11 @@

      Allsky Setting A (partial) typical page is below. The entries highlighted in yellow are Advanced entries that aren't changed very often; to see them, click on the - -button on the bottom of the page. +Show advanced options +button on the top of the page. +Note that in the screenshot below, that button was already pressed so the +Hide advanced options +button is shown instead, as well as some advanced settings.

      AllskySettings @@ -121,7 +124,7 @@

      Allsky Setting

      - - + @@ -153,7 +160,10 @@

      Allsky Setting

      + increasing this produces brighter images, but with more noise. +
      The default daytime gain is the minimum possible for the camera since + daytime images are normally bright and don't need any additional gain. + @@ -162,8 +172,8 @@

      Allsky Setting Increasing the bin results in smaller, lower-resolution images and reduces the need for long exposure. Look up your camera specifications to know what values are supported. -
      This setting is usually only changed during the day for testing. -
      On CCD camera, binning normally produces brighter images. +
      During the day this setting is usually only changed for testing. +
      Binning on CCD cameras normally produces brighter images. CMOS camera may, or may not produce brighter images, depending on the camera model.
      @@ -201,8 +211,8 @@

      Allsky Setting

      @@ -224,11 +234,14 @@

      Allsky Setting

      - + - + @@ -246,7 +259,7 @@

      Allsky Setting

      - + @@ -351,7 +364,7 @@

      Allsky Setting config.sh. With RPi cameras these settings decrease resolution of the full-sensor image, then increase the resolution, thereby negating the changes. - There is no reason to set the Width and +
      There is no reason to set the Width and Height with RPi cameras.

      @@ -374,12 +387,18 @@

      Allsky Setting For PNG, this is the amount of compression - 0 for no compression (but quicker to save) to 9 for highest compression (but slowest to save). If you use very short delays between pictures you may want to play with these - numbers to ensure the quickest delay possible. + numbers to get the quickest delay possible.

      - + @@ -409,7 +428,7 @@

      Allsky Setting

      - @@ -485,6 +505,16 @@

      Allsky Setting when done, re-run the installation script selecting the locale you just installed.

      + + + + - @@ -701,7 +733,7 @@

      Allsky Setting you may need to set this field to <Pi name>/current/tmp/image.jpg or whatever's in the imageName field of your website's - configuration.js file. + configuration.json file. Be careful of using "http" versus "https", and after enabling Show On Map, look at the map to ensure you can see your image. If your camera is not accessible on the Internet or you do not want the image @@ -711,8 +743,7 @@

      Allsky Setting

      + You can put any level of detail you want. @@ -744,6 +775,9 @@

      Allsky Setting

      @@ -765,13 +799,15 @@

      Editor We

      Editor

      -Files that can be edited include config.sh, +This page allows you to edit the config.sh file, +and if you have a local and/or remote Allsky Website installed, the ftp-settings.sh, -and if you have the Allsky Website installed, a local and/or remote -configuration.json file. +configuration.json, and +remote_configuration.json files can also be edited. Further, if you have a endOfNight_additionalSteps.sh file, it can also be edited. -

      + +
      The endOfNight_additionalSteps.sh file will no longer be supported in the next version of Allsky. If you use this file, please move your code to the Script module @@ -782,15 +818,18 @@

      Editor We

      -The tables below describe the settings in the -config.sh and -ftp-settings.sh files; -settings in the configuration.json files are described in the -Allsky Website Settings page. +The table below describes the settings in the +config.sh file. +
      +Settings in the ftp-settings.sh file are described in the +ftp-settings.sh settings page. +
      +Settings in the configuration.json files are described in the +Allsky Website Settings page.

      -Information on the color scheme used by the Editor is -here . +Information on the color scheme used by the Editor in the screenshot above is +here .

      config.sh settings

      @@ -1224,281 +1263,6 @@

      config.sh settings

      -

      ftp-settings.sh settings

      -

      -In order to upload files to your Allsky Website (on the Pi and/or a remote server), -connection details must be specified by clicking on the WebUI's -Editor page, then selecting -ftp-settings.sh in the drop-down list at the bottom of the page. -

      - -
      -

      -Notes: -

        -
      • - In the text below, "local" refers to an Allsky Website that's on the Pi - and "remote" refers to an Allsky Website not on the Pi. -
        - Directories on the Pi are created during Allsky Website installation, but - YOU must create the remote directories. -
        -
      • - The WEB_*_DIR - settings below (e.g., WEB_IMAGE_DIR) - are only used if you have an Allsky Website on the Pi - AND a remote Allsky Website, and you want files copied to both locations. - In this case, see the example at the end of the table for what settings to use. -
      • - By default, the destination file name is the same as the file being uploaded. - The *_DESTINATION_NAME settings below are used - to specify a DIFFERENT destination name. - For example, if the file being uploaded is - allsky-20210710.mp4 you may want it - called allsky.mp4 on the remote web server - so the name is always the same. - In that case, set - VIDEOS_DESTINATION_NAME="allsky.mp4" - (don't forget the file name extension like .mp4, .jpg, etc.). - If you want the destination file name to be the same as what's being uploaded, - leave the *_DESTINATION_NAME blank. -
      -

      -
      - -

      Mean Target 0.5(RPi only) The target mean brightness level when + The target mean brightness level when Auto-Exposure is on. Ranges from 0.0 (pure black) to 1.0 (pure white). Best used when both Auto-Exposure and @@ -129,7 +132,11 @@

      Allsky Setting

      Brightness CDThis setting changes the amount of light in images.This setting changes the amount of light in images. +
      This settings has been deprecated and will be + removed in a future release. + Use Mean Target to adjust the brightness instead. +
      Delay 5000Gain is similar to ISO on regular cameras. When Auto-Gain is on, this value is used as a starting gain. When Auto-Gain is off, - increasing this produces brighter images, but with more noise.
      Binning 1x1
      Tuning File No (RPi on Bullseye only) Name of the optional daytime tuning file. - See this documentation for more information. + See this documentation for more information.
      Mean Target 0.2(RPi only)
      Brightness CDThis settings has been deprecated and will be + removed in a future release. + Use Mean Target to adjust the brightness instead. +
      Delay 10
      Gain CDThe default is one-half the maximum for the camera.
      Binning 1x1
      Image Height
      Mean Threshold Yes(RPi only) When using Mean Target, - this specifies how close the target brightness should be to the Target.When using Mean Target, + this specifies how close (plus or minus) the target brightness should be to the + Mean Target. +
      For example, if Mean Target is + 0.4 and + Mean Threshold is 0.1 + then the target brightness ranges from 0.3 to 0.5 +
      Auto USB Bandwidth Yes
      Consistent Delays Between Images YesEnable to force the time between the start of exposures to be a consistent length + Enable this to force the time between the start of exposures to be a consistent length (Max Auto-Exposure + Delay). Doing this will result in timelapse video frames being equally spaced, for example, every 90 seconds, regardless of how long an individual frame's exposure is. @@ -447,8 +466,9 @@

      Allsky Setting

      No Enable to take dark frames which are use to decrease noise in images.
      - See this page - for an in-depth explanation of dark frames, including how to take and use them. + See the + in-depth explanation + of dark frames, including how to take and use them.
      Use Dark Frames
      New Exposure AlgorithmNo(ZWO only) Activate to use a auto-exposure algorithm at night. + Initial testing indictes the images taken during the day-to-night transition + as well as at night have better exposures. + If you use this, please add a Discussion item describing your results - good or bad. + We need the feedback. +
      If this provides better results overall it will be the default in the next release. +
      Histogram Box 500 500 50 50 (ZWO only) X and Y size of histogram box in pixels and the middle point of @@ -499,11 +529,13 @@

      Allsky Setting

      Determines the amount of output in the log file. Log entries can also be viewed with journalctl -u allsky.
      0 outputs error messages only. + 4 outputs a LOT of messages and generally should + only be used if an Allsky developers directs you to.
      Version 0.8 Exposure Yes(ZWO only) Determines if the new version 0.8 exposure method is used (video capture stops between images). + (ZWO only) Determines if the Allsky version 0.8 exposure method is used (video capture stops between images). This decreases the sensor temperature between 5 - 15 degrees Celsius. If you see ASI_ERROR_TIMEOUTs in the log file, try turning this off. See Issue 417.
      Location AW The location of your camera. - You can put any level of detail you want, - but there's no need to enter the country since it will be obvious by looking at the map.
      Owner AW The owner of the camera - your name, an association name, an observatory, etc.
      The type of camera you are using: ZWO or RPi (which includes the HQ and compatible models).
      This is initially set during Allsky installation. +
      If you replace a camera with another one of the same type + (e.g., a ZWO ASI120 with a ZWO ASI290) + select Refresh in the drop-down.
      Camera Model
      - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
      SettingDefaultDescription
      PROTOCOL How the file should be uploaded: -
        -
      • local - - copies the file to a local Allsky Website. -
      • ftp - - uses File Transfer Protocol (FTP) to upload to a remote server. -
      • ftps - - uses secure File Transfer Protocol (FTPs) to upload to a remote server. -
        This is preferred over the insecure - ftp. -
      • sftp - - uses SSH file transfer to upload to a remote server. -
      • scp - - uses secure cp (copy) to copy the file to a remote server. -
      • S3 - - copies the file to an Amazon Web Services (AWS) server. -
      • gcs - - copies the file to a Google Cloud Storage (GCS) server. -
      -
      image.jpg Settings
      IMAGE_DIR The remote directory where the current image should go.
      WEB_IMAGE_DIR The local directory where the current image should go.
      Timelapse Settings
      VIDEOS_DIR The remote directory where the timelapse video should go.
      VIDEOS_DESTINATION_NAME Remote name of the timelapse video file. - If not specified it's the same name as the file being uploaded.
      WEB_VIDEOS_DIR Location on the Pi to copy the timelapse video to.
      Keogram Settings
      KEOGRAM_DIR The remote directory where the keogram image should go.
      KEOGRAM_DESTINATION_NAME Remote name of the keogram image file. - If not specified it's the same name as the file being uploaded.
      WEB_KEOGRAM_DIR Location on the Pi to copy the keogram file to.
      Startrails Settings
      STARTRAILS_DIR The remote directory where the startrails image should go.
      STARTRAILS_DESTINATION_NAME Remote name of the startrails file. - If not specified it's the same name as the file being uploaded.
      WEB_STARTRAILS_DIR Location on the Pi to copy the startrails file to.
      ftp, ftps, sftp, and scp Settings
      REMOTE_HOST Remote server DNS name or IP address. - If you don't know it, ask your service provider. -
      ftp, ftps, and sftp Settings
      REMOTE_USER Your remote user name.
      REMOTE_PASSWORD Your ftp / ftps / sftp password.
      REMOTE_PORT An optional port number for ftp and ftps. Is rarely needed.
      LFTP_COMMANDS An optional colon-separated (;) list of commands needed by the lftp command. - See this page - for example commands to enter into LFTP_COMMANDS. -
      scp Settings
      SSH_KEY_FILE Path to the SSH key file. -
      - You need to set up SSH key authentication on your server. - First, generate an SSH key on your Pi: -
      ssh-keygen -t rsa
      - When prompted, leave the default filename, and use an empty passphrase. - Then, copy the generated key to your server: -
      ssh-copy-id remote_username@server_ip_address
      - The private SSH key will be stored in - ~/.ssh (default filename is id_rsa) -
      S3 Settings
      - You need to install the AWS Command-Line Interface (CLI): -
      -sudo apt-get install python3-pip
      -pip3 install awscli --upgrade --user
      -export PATH=/home/pi/.local/bin:$PATH
      -aws configure
      -
      - When prompted, enter a valid access key ID, Secret Access Key, and Default region name, - for example, (e.g. "us-west-2"). - Set the Default output format to "json" when prompted. -
      AWS_CLI_DIR/home/pi/.local/binDirectory on the Pi where the AWS tools are installed. -
      If you used a different PATH setting above, change this setting to match it. -
      S3_BUCKETallskybucketName of S3 Bucket where the files will be uploaded - (must be in Default region specified above). - You may want to turn off or limit bucket versioning to avoid consuming lots of - space with multiple versions of the "image.jpg" files. -
      S3_ACLprivateS3 Access Control List (ACL). - If you want to serve your uploaded files vis http(s), - change this to public-read. - You will need to ensure the S3 bucket policy is configured to allow public access to - objects with a public-read ACL. -
      You may need to set a CORS policy in S3 if the files are to be accessed by - Javascript from a different domain. -
      GCS Settings
      - You need to install the gsutil command which is part of the Google Cloud SDK. - See installation instructions - here. -
      - NOTE: The gsutil command must be installed somewhere in the standard $PATH, - usually in /usr/bin. - Make sure you authenticate the cli tool with the correct user as well. -
      GCS_BUCKETallskybucketName of S3 Bucket where the files will be uploaded.
      GCS_ACLprivateGCS Access Control List. - You can use any one of the predefined ACL rules found - here. - To access files over https, set this to publicRead. -
      - -

      Settings for various combinations

      -
        -
      1. If you have the Allsky Website only on your Pi, use these settings: -
        -PROTOCOL="local"
        -IMAGE_DIR=""
        -VIDEOS_DIR="${ALLSKY_WEBSITE}/videos"
        -KEOGRAM_DIR="${ALLSKY_WEBSITE}/keograms"
        -STARTRAILS_DIR="${ALLSKY_WEBSITE}/startrails"
        -
        - -
      2. - If you have an Allsky Website only on a remote server, use these settings. - For the sake of this example, assume your top-level directory on the server is - /allsky: -
        -PROTOCOL="sftp"		# or another PROTOCOL
        -IMAGE_DIR="/allsky"
        -VIDEOS_DIR="/allsky/videos"
        -KEOGRAM_DIR="/allsky/keogram"
        -STARTRAILS_DIR="/allsky/startrails"
        -
        - -
      3. - If you have an Allsky Website on your Pi AND on a remote server, use these settings. - For the sake of this example, assume your top-level directory on the server is - /allsky: -
        -PROTOCOL="sftp"		# or another PROTOCOL
        -IMAGE_DIR="/allsky"
        -WEB_IMAGE_DIR=""
        -VIDEOS_DIR="/allsky/videos"
        -WEB_VIDEOS_DIR="${ALLSKY_WEBSITE}/videos"
        -KEOGRAM_DIR="/allsky/keogram"
        -WEB_KEOGRAM_DIR="${ALLSKY_WEBSITE}/keograms"
        -STARTRAILS_DIR="/allsky/startrails"
        -WEB_STARTRAILS_DIR="${ALLSKY_WEBSITE}/startrails"
        -
        -
      - -

      Amazon Lightsail

      -If you are using SSH Key with Amazon's Lightsail, -copy the ssh-key.pem file to your Pi, -for example, in ~, -then execute -
      chmod 400 ~/ssh-key.pem
      -and set: -
      -PROTOCOL="sftp"
      -REMOTE_HOST="remote host name"
      -REMOTE_USER="remote user name"
      -REMOTE_PASSWORD="n/a"
      -LFTP_COMMANDS="set sftp:connect-program 'ssh -a -x -i /home/pi/ssh-key.pem'"
      -
      - -
      - - + +

      configuration.json files

      -The Allsky Website configuration file has two sections: +This section describes the settings in the +configuration.json (local Allsky Website) and +remote_configuration.json (remote Allsky Website) files. +

      +
      +

      +The settings in both files are identical (although their values may differ) +and they are split into two sections:

      1. "config" - settings for the liveview image and constellation overlay.
      2. "homePage" - settings to change the look and feel - of the Website's home page including the icons on the left side and the - information popout on the right side. + of the Website's home page including the icons on the left side, + the information popout on the right side, an optional background image, etc.

      -Any setting whose default is XX_need_to_update_XX needs to be updated prior -to using the Website. -The latitude and longitude also need to be updated if not already updated. -

      -

      -The sections below list all the settings, their default values, and a description. -Information on the color scheme used by the Editor is -here . +Any setting whose value is XX_need_to_update_XX +needs to be updated prior to using the Website.

      -

      "config" settings

      +

      config settings

      -
      -Legend: +
      +Legend:
        -
      • Values for setting names with after them +
      • Values for setting names with "" after them are sent to the Allsky Map server if you camera is on the map. -
      • Setting names with - after them impact the virtual sky overlay. +
      • Setting names with "" after them + impact the virtual sky overlay. A complete list of virtual-sky based options is here.
      • Values marked with are automatically set during - installation based on your settings and your Pi model, but can be overridden. + installation based on your WebUI settings and your Pi model, but can be overridden.
      It's important to update your settings in the WebUI before installing the Allsky Website so you only have to update them once.
      +

      @@ -104,10 +421,11 @@

      "config" settings

      - + @@ -424,7 +742,7 @@

      "config" settings

      -

      "homePage" settings

      +

      homePage settings


      imageName/current/tmp/image.jpg/current/tmp/image.jpg (local) or + image.jpg (remote) + The image uploaded from your allsky camera. - On a remote web server this should normally be - image.jpg. +
      Normally should not be changed.
      @@ -657,6 +975,9 @@

      "homePage" settings

      +
      + +
    diff --git a/html/documentation/troubleshooting/ZWOCameras.html b/html/documentation/troubleshooting/ZWOCameras.html index 2c92f8783..08421e71b 100644 --- a/html/documentation/troubleshooting/ZWOCameras.html +++ b/html/documentation/troubleshooting/ZWOCameras.html @@ -108,8 +108,8 @@

    ASI_ERROR_TIMEOUT Errors

    Try increasing and decreasing it, and try turning Auto USB Bandwidth on and off. You will have to click on the - - button at the bottom of the page in order to view the USB options. + Show advanced options + button at the top of the page in order to view the USB options.
  • See if the system is in under-voltage mode or is throttling which could lead to insufficient power getting to the camera. In the WebUI, go to the System page; @@ -126,11 +126,10 @@

    ASI_ERROR_TIMEOUT Errors

  • Revert to the pre-0.8 exposure method. In the WebUI, click on the Camera Settings link. At the bottom of the page click on the - + Show advanced options button and look for the Version 0.8 Exposure setting and toggle it then click on the - - button. + Save changes button.
  • The above changes work for almost everyone. If they don't work for you, follow the instructions on the Reporting Issues diff --git a/html/documentation/troubleshooting/image-flicker.html b/html/documentation/troubleshooting/image-flicker.html index 8effbf841..d26729f33 100644 --- a/html/documentation/troubleshooting/image-flicker.html +++ b/html/documentation/troubleshooting/image-flicker.html @@ -59,8 +59,8 @@

    Daytime only flicker - ZWO cameras

    sudo truncate -s 0 /var/log/allsky.log In the WebUI change the Debug Level to -4 then click on -. +4 then click on the +Save changes button. Let it run for a couple minutes so it includes the flickering, then upload /var/log/allsky.log.

    diff --git a/html/documentation/troubleshooting/reportingIssues.html b/html/documentation/troubleshooting/reportingIssues.html index 71948443c..4fb57e863 100644 --- a/html/documentation/troubleshooting/reportingIssues.html +++ b/html/documentation/troubleshooting/reportingIssues.html @@ -48,7 +48,7 @@

    Change Debug Level to 4 in the WebUI then click on the - button. +Save changes button. This will restart the service.

    diff --git a/html/includes/allskySettings.php b/html/includes/allskySettings.php index c52ef57fd..cb9083c73 100644 --- a/html/includes/allskySettings.php +++ b/html/includes/allskySettings.php @@ -1,7 +1,4 @@ " . $option['label'] . " is empty"; - else - $errorMsg .= ", " . $option['label'] . " is empty"; - $numErrors++; + $msg = "<$span>$lab is empty"; + $status->addMessage($msg, 'danger', false); $ok = false; + + } else if ($type !== null) { + $msg = ""; + // $value will be of type string, even if it's actually a number, + // and only is_numeric() accounts for types of string. + if ($type === "integer" || $type == "percent") { + if (! is_numeric($value) || ! is_int($value + 0)) + $msg = "without a fraction"; + } else if ($type === "float") { + if (! is_numeric($value) || ! is_float($value + 0.0)) + $msg = "with, or without, a fraction"; + } + if ($msg !== "") { + $msg2 = "<$span>$lab must be a number $msg."; + $msg2 .= " You entered: <$spanValue>$value"; + $status->addMessage($msg2, 'danger', false); + $ok = false; + } } } } - // Add the key/value pair to the array so we can see if it changed. - $settings[$key] = str_replace("'", "'", $value); + if ($ok) { + $settings[$key] = str_replace("'", "'", $value); - if ($key === $debugLevelName && $value >= 4) { - $debugArg = "--debug"; + if ($key === $debugLevelName && $value >= 4) { + $debugArg = "--debug"; + } } } } -// TODO: makeChanges.sh should probably come first because if it fails, we don't want -// to update the settings file. + $msg = ""; if ($ok) { - if ($somethingChanged || $lastChanged === null) { + if ($somethingChanged || $lastChanged === "") { if ($newCameraType !== "" || $newCameraModel !== "" || $newCameraNumber != "") { $msg = "If you change Camera Type, Camera Model,"; - $msg .= " or Camera Number you cannot change anything else. No changes made."; + $msg .= " or Camera Number you cannot change anything else."; + $status->addMessage($msg, 'danger', false); $ok = false; } else { // Keep track of the last time the file changed. @@ -141,7 +163,10 @@ function DisplayAllskyConfig(){ } } else { if ($newCameraType !== "") { - $msg .= "Camera Type changed to $newCameraType"; + if ($refreshingCameraType) + $msg .= "Camera Type $newCameraType refreshed"; + else + $msg .= "Camera Type changed to $newCameraType"; } if ($newCameraModel !== "") { if ($msg !== "") $msg = "
    $msg"; @@ -153,27 +178,31 @@ function DisplayAllskyConfig(){ } if ($msg === "") - $msg = "No settings changed (file not updated)"; + $msg = "No settings changed (but timestamp updated)"; } } - // 'restart' is a checkbox: if check, it returns 'on', otherwise nothing. - $doingRestart = getVariableOrDefault($_POST, 'restart', false); - if ($doingRestart === "on") $doingRestart = true; - if ($ok) { + // 'restart' is a checkbox: if check, it returns 'on', otherwise nothing. + $doingRestart = getVariableOrDefault($_POST, 'restart', false); + if ($doingRestart === "on") $doingRestart = true; + if ($changes !== "") { // This must run with different permissions so makeChanges.sh can // write to the allsky directory. + $moreArgs = ""; if ($doingRestart) - $restarting = "--restarting"; - else - $restarting = ""; + $moreArgs .= " --restarting"; + if ($newCameraType !== "") { + $moreArgs .= " --cameraTypeOnly"; + } + $CMD = "sudo --user=" . ALLSKY_OWNER; - $CMD .= " " . ALLSKY_SCRIPTS . "/makeChanges.sh $debugArg $restarting $changes"; + $CMD .= " " . ALLSKY_SCRIPTS . "/makeChanges.sh $debugArg $moreArgs $changes"; # Let makeChanges.sh display any output echo ''; - $ok = runCommand($CMD, "", "success"); + // false = don't add anything to the message + $ok = runCommand($CMD, "", "success", false); } if ($ok) { @@ -182,18 +211,13 @@ function DisplayAllskyConfig(){ // runCommand displays $msg. runCommand("sudo /bin/systemctl reload-or-restart allsky.service", $msg, "success"); } else { - $msg .= " but Allsky NOT restarted."; + $msg .= "; Allsky NOT restarted."; $status->addMessage($msg, 'info'); } } } else { // not $ok - if ($doingRestart) - $msg = ", and Allsky NOT restarted."; - if ($numErrors > 0) { - $msg = "Settings NOT saved due to $numErrors errors: $errorMsg."; - } - $status->addMessage($msg, 'danger'); + $status->addMessage("Settings NOT saved due to errors above.", 'info', false); } } else { $status->addMessage('Unable to save settings - session timeout.', 'danger'); @@ -295,6 +319,27 @@ function toggle_advanced() ?>

    + + +
    + + + +
    +
    + Restart Allsky after saving changes? +

      +
    +
    + + + "> "; - echo "$description"; - echo ""; + // Not sure how to display the header with a background color with 10px + // of white above and below it using only one . + echo ""; + echo ""; + echo "$description"; + echo ""; + echo ""; } else { echo ""; // Show the default in a popup - if ($type == "checkbox") { + if ($type == "boolean") { if ($default == "0") $default = "No"; else $default = "Yes"; } elseif ($type == "select") { @@ -413,12 +462,12 @@ function toggle_advanced() break; } } - if ($default !== "") - $popup = "Default=$default"; - else - $popup = ""; + $popup = ""; + if ($default !== "") $popup .= "Default=$default"; if ($minimum !== "") $popup .= "\nMinimum=$minimum"; if ($maximum !== "") $popup .= "\nMaximum=$maximum"; + if ($type == "integer" || $type == "percent") $popup .= "\nWhole numbers only"; + if ($type == "float") $popup .= "\nFractions allowed"; if ($type == "widetext") $span="rowspan='2'"; else $span=""; @@ -441,7 +490,8 @@ function toggle_advanced() // May want to consider having a symbol next to the field // that has the popup. echo ""; - if ($type == "text" || $type == "number" || $type == "readonly"){ +// TODO: add percent sign for "percent" + if ($type == "text" || $type == "integer" || $type == "float" || $type == "percent" || $type == "readonly"){ if ($type == "readonly") { $readonly = "readonly"; $t = "text"; @@ -449,8 +499,9 @@ function toggle_advanced() $readonly = ""; // Browsers put the up/down arrows for numbers which moves the // numbers to the left, and they don't line up with text. - // Plus, they don't accept decimal points in "number". - if ($type == "number") $type = "text"; + // Plus, they don't accept decimal points in "float". + if ($type == "integer" || $type == "float" || $type == "percent") + $type = "text"; $t = $type; } echo "\n\t"; - } else if ($type == "checkbox"){ + } else if ($type == "boolean"){ echo "\n\t
    "; echo "\n\t -if ($formReadonly != "readonly") { ?> -
    - - - -

    Restart Allsky after saving changes?

     
    -
    -
    diff --git a/html/includes/createAllskyOptions.php b/html/includes/createAllskyOptions.php index e1d43cb0f..96df3e3d3 100755 --- a/html/includes/createAllskyOptions.php +++ b/html/includes/createAllskyOptions.php @@ -18,16 +18,30 @@ function get_generic_name($s) { return substr($s, 5); return $s; } +// These values need to be looked up in the CC file using their generic name. +function is_generic_value($v) { + if ($v === null) return false; + + if (substr($v, 0, 1) === "_") + return true; + + return false; +} +// These values need to be looked up in the CC file using their full name. +function is_specific_value($v) { + if ($v === null) return false; + + if (substr($v, 0, 4) === "day_" || substr($v, 0, 6) === "night_") + return true; + + return false; +} // Get a camera control. Return true if it exists, false if it doesn't. // If it exists, set the min, max, and default. function get_control($array, $setting, &$min, &$max, &$default) { - $i = 0; -//x echo "Looking for control [$setting]: "; foreach ($array as $cc) { - $i++; if ($cc["argumentName"] === $setting) { -//x echo "match at number $i\n"; $min = getVariableOrDefault($cc, "MinValue", null); $max = getVariableOrDefault($cc, "MaxValue", null); $default = getVariableOrDefault($cc, "DefaultValue", null); @@ -40,7 +54,7 @@ function get_control($array, $setting, &$min, &$max, &$default) { // If a field is null that means it wasn't in the repo file, // so don't add it to the options string. // We need this because we look for all fields in a setting. -function add_non_null_field($a, $f, $setting) { // array, field, name_of_setting +function add_non_null_field($a, $f, $setting) { // array, field name, name_of_setting $value = getVariableOrDefault($a, $f, null); if ($value === null) return; @@ -79,34 +93,52 @@ function add_field($f, $v, $setting) { // field, value, name_of_setting global $options_str; $options_str .= "$q$f$q : "; // field name - // Do not add value if a string since we need to check if it needs to be replaced + // Do not add value if it's a string since we need to check if it needs to be replaced if (! add_value($v, false)) { - // If the setting is a day/night one, e.g., "daybin", get just the "bin" portion. - $setting = get_generic_name($setting); - - // Check if the value is a placeholder, like "bin_min" for the "bin" setting. - // These are the only fields that have placeholders. - // The "options" field is handles in add_options_field() since it's value is an array. + if ($debug > 1) { + // It's hard to read the output with really long strings. + if (strlen($v) > 50) $vv = substr($v, 0, 50) . "..."; + else $vv = $v; + echo " '$f', v='$vv'"; + } + + // Check if the value is a generic placeholder, like "_min". + // The "options" field is handled in add_options_field() since it's value is an array. // The "display" field was handled earlier. -if ($debug > 1) echo "Setting '$setting', field '$f', v='$v'\n"; + if (is_generic_value($v) || is_specific_value($v)) { + $searchCC = true; + } else { + $searchCC = false; + } + + if ($searchCC) { + // For generic values, if the setting is a day/night one, e.g., "dayexposure", + // get just the "exposure" portion. + // For specific values e.g., "daymean" : "day_default", + // need to look up "daymean" in the CC file, + // not "mean" like we do for generic values - if (get_control($cc_controls, $setting, $min, $max, $default)) { -if ($debug > 1) echo " >>> found in controls list\n"; - if ($f === "minimum") { - if ($v === $setting . "_min") { + if (is_generic_value($v)) { + $setting = get_generic_name($setting); + } + if (get_control($cc_controls, $setting, $min, $max, $default)) { + $vReset = false; + if ($f === "minimum") { $v = $min; - } - } else if ($f === "maximum") { - if ($v === $setting . "_max") { + $vReset = true; + } else if ($f === "maximum") { $v = $max; - } - } else if ($f === "default") { - if ($v === $setting . "_default") { -if ($debug > 1) echo " >>>>> Setting '$setting', field '$f' _default=[$default]\n"; + $vReset = true; + } else if ($f === "default") { $v = $default; + $vReset = true; + } + if ($debug > 1) { + if ($vReset) echo ", RESET v='$v'"; } } } + if ($debug > 1) echo "\n"; $options_str .= "$q$v$q"; } @@ -218,7 +250,7 @@ function add_options_field($field, $options, $setting) { $force = false; // force creation of settings file even if it already exists? foreach ($options as $opt => $val) { - if ($debug > 1) echo " Argument $opt = $val\n"; + if ($debug > 1 || $opt === "debug") echo " Argument $opt $val\n"; if ($opt === "debug") $debug++; @@ -286,11 +318,25 @@ function add_options_field($field, $options, $setting) { // display [0/1] // checkchanges [0/1] // optional [0/1] - // nullOK [0/1] + // generic [0/1] // advanced [0/1] (last, so no comma after it) -// Create options file +// ================== Create options file + +// A "generic" value is one that's the same for day and night, e.g., the minimum value +// for the "dayexposure" and "nightexposure". +// These are often specified by the camera and have an "argumentName" in the CC +// file without the "day" or "night", e.g., "exposure. + +// Field values that begin with "_", e.g., "_default" are generic placeholders; their +// actual values need to be determined by looking in the CC file for the generic name. +// The repo options file will typically have "_" followed by the field name, +// e.g., "_default" for the "default" field, but we only check if the first char is "_". + +// Field values that being with "day_" or "night_", e.g., "day_default" have +// different values for day and night in the CC file, e.g., default value for day +// and night exposure. $options_str = "[\n"; foreach ($repo_array as $repo) { @@ -309,30 +355,25 @@ function add_options_field($field, $options, $setting) { if ($debug > 1) echo "Processing setting [$name]: "; - // Before adding the setting, make sure the "display field says we can. - // Typically the value will be 1 (can display) or a placeholder. + // Before adding the setting, make sure the "display" field says we can. + // The value will be 1 (can display) or 0 (don't display), or a placeholder. // It should normally not be missing, but check anyhow. $display = getVariableOrDefault($repo, "display", null); - if ($display === null) { - if ($debug > 1) echo "display field=null\n"; + if ($display === null || $display === 0) { + if ($debug > 1) echo " display field is null or 0\n"; continue; } - if ($display !== 1) { - // should be a placeholder - $n = get_generic_name($name); - if ($display === $n . "_display") { - if ($debug > 1) echo "display=$display."; - if (! get_control($cc_controls, $n, $min, $max, $default)) { - if ($debug > 1) echo " <<<<< NOT SUPPORTED >>>>>\n"; - // Not an error - just means this isn't supported. - continue; - } - if ($debug > 1) echo "\n"; - $repo["display"] = 1; // a control exists for it, so display the setting. + if (is_generic_value($display)) { + // Is a placeholder - need to check if the setting is in the CC file. + // If not, don't output this setting. + if (! get_control($cc_controls, get_generic_name($name), $min, $max, $default)) { + if ($debug > 1) echo " <<<<< NOT SUPPORTED >>>>>\n"; + // Not an error - just means this isn't supported. + continue; } - } elseif ($debug > 1) { - echo "standard setting.\n"; + $repo["display"] = 1; // a control exists for it, so display the setting. } + if ($debug > 1) echo "\n"; // Have to handle camera type and model differently because the defaults // might not be what we want. @@ -340,6 +381,8 @@ function add_options_field($field, $options, $setting) { $repo["default"] = $cameraType; elseif ($name === "cameraModel") $repo["default"] = $cameraModel; + elseif ($name === "camera") + $repo["default"] = "$cameraType $cameraModel"; $options_str .= "{\n"; add_non_null_field($repo, "name", $name); @@ -353,7 +396,7 @@ function add_options_field($field, $options, $setting) { add_non_null_field($repo, "display", $name); add_non_null_field($repo, "checkchanges", $name); add_non_null_field($repo, "optional", $name); - add_non_null_field($repo, "nullOK", $name); + add_non_null_field($repo, "generic", $name); add_non_null_field($repo, "advanced", $name); $options_str .= "},\n"; } @@ -366,14 +409,34 @@ function add_options_field($field, $options, $setting) { exit(6); } -// Optionally create a basic "settings" file with the default for this camera type/model. + +// ================== Create settings file + +// If a $settings_file was passed in, create a "settings" file. +// If this camera type/model is already known, use that file, +// otherwise create a "settings" file with defaults for this camera. +// However, if there's an old settings file port its generic fields to the new file. if ($settings_file !== "") { - $options_array = json_decode($options_str, true); + // Determine the name of the camera type/model-specific file. + $pieces = explode(".", basename($settings_file)); // e.g., "settings.json" + $FileName = $pieces[0]; // e.g., "settings" + $FileExt = $pieces[1]; // e.g., "json" + // e.g., "settings_ZWO_ASI123.json" + $cameraSpecificSettingsName = $FileName . "_$cameraType" . "_$cameraModel.$FileExt"; + $fullSpecificFileName = dirname($settings_file) . "/$cameraSpecificSettingsName"; + if ($debug > 0) { + $e = file_exists($fullSpecificFileName) ? "yes" : "no"; + echo "Camera-specific settings file exists ($e): $fullSpecificFileName.\n"; + } - // If the file exists, it's a generic link to a camera-specific named file. + // If the settings file exists, it's a generic link to a camera-specific named file. // Remove the link because it points to a prior camera. + $settings_array = null; if (file_exists($settings_file)) { + $errorMsg = "ERROR: Unable to process prior settings file '$settings_file'."; + $settings_array = get_decoded_json_file($settings_file, true, $errorMsg); + if ($debug > 0) echo "Removing $settings_file.\n"; if (! unlink($settings_file)) { echo "ERROR: Unable to delete $settings_file.\n"; @@ -381,22 +444,12 @@ function add_options_field($field, $options, $setting) { } } - // Determine the name of the camera type/model-specific file. - $pieces = explode(".", basename($settings_file)); // e.g., "settings.json" - $FileName = $pieces[0]; // e.g., "settings" - $FileExt = $pieces[1]; // e.g., "json" - // e.g., "settings_ZWO_ASI123.json" - $cameraSpecificSettingsName = $FileName . "_$cameraType" . "_$cameraModel.$FileExt"; - $fullSpecificFile = dirname($settings_file) . "/$cameraSpecificSettingsName"; - if ($debug > 0) { - $e = file_exists($fullSpecificFile) ? "yes" : "no"; - echo "Camera-specific settings file exists ($e): $fullSpecificFile.\n"; - } // If there isn't a camera-specific file, create one. - if ($force || ! file_exists($fullSpecificFile)) { - // For each item in the options file, write the name and default value. + if ($force || ! file_exists($fullSpecificFileName)) { + // For each item in the options file, write the name and a value. $contents = "{\n"; + $options_array = json_decode($options_str, true); foreach ($options_array as $option) { $type = getVariableOrDefault($option, 'type', ""); if ($type == "header") continue; // don't put in settings file @@ -404,21 +457,32 @@ function add_options_field($field, $options, $setting) { if ($display === 0) continue; $name = $option['name']; - $default = getVariableOrDefault($option, 'default', ""); - if ($debug > 1) echo ">> $name = [$default]\n"; + // If it's a generic setting, use it's prior value if it exists. + if (getVariableOrDefault($option, 'generic', 0) !== 0 && $settings_array !== null) { + $val = getVariableOrDefault($settings_array, $name, null); + } else { + $val = null; + } + + if ($val === null) { + $val = getVariableOrDefault($option, 'default', ""); + if ($debug > 1) echo ">> default $name = [$val]\n"; + } else { + if ($debug > 1) echo ">> generic $name = [$val]\n"; + } // Don't worry about whether or not the default is a string, number, etc. - $contents .= "\t\"$name\" : \"$default\",\n"; + $contents .= "\t\"$name\" : \"$val\",\n"; } // This comes last so we don't worry about whether or not the items above // need a trailing comma. $contents .= "\t\"XX_END_XX\" : 1\n"; $contents .= "}\n"; - if ($debug > 0) echo "Creating camera-specific settings file: $fullSpecificFile.\n"; - $results = updateFile($fullSpecificFile, $contents, $cameraSpecificSettingsName, true); + if ($debug > 0) echo "Creating camera-specific settings file: $fullSpecificFileName.\n"; + $results = updateFile($fullSpecificFileName, $contents, $cameraSpecificSettingsName, true); if ($results != "") { - echo "ERROR: Unable to create $fullSpecificFile.\n"; + echo "ERROR: Unable to create $fullSpecificFileName.\n"; exit(8); } @@ -426,12 +490,12 @@ function add_options_field($field, $options, $setting) { // There IS a camera-specific file for the new camera type so we // don't need to do anything special. // The generic name will be linked to the specific name below. - echo "Using existing $fullSpecificFile.\n"; + echo "Using existing $fullSpecificFileName.\n"; } - if ($debug > 0) echo "Linking $fullSpecificFile to $settings_file.\n"; - if (! link($fullSpecificFile, $settings_file)) { - echo "ERROR: Unable to link $fullSpecificFile to $settings_file.\n"; + if ($debug > 0) echo "Linking $fullSpecificFileName to $settings_file.\n"; + if (! link($fullSpecificFileName, $settings_file)) { + echo "ERROR: Unable to link $fullSpecificFileName to $settings_file.\n"; exit(9); } } diff --git a/html/includes/dashboard_WLAN.php b/html/includes/dashboard_WLAN.php index f90369b10..f2e27c8b3 100644 --- a/html/includes/dashboard_WLAN.php +++ b/html/includes/dashboard_WLAN.php @@ -1,12 +1,48 @@ +
    +
    +
    +
    WLAN Dashboard
    + $v) { + process_WLAN_data($int, $v, $num++); + } +?> + +
    +
    +
    + 1) { + echo "
    "; + } // $interface_output is sent and the other variables are returned. parse_ifconfig($interface_output, $strHWAddress, $strIPAddress, $strNetMask, $strRxPackets, $strTxPackets, $strRxBytes, $strTxBytes); @@ -50,18 +86,15 @@ function DisplayDashboard_WLAN($interface) { // $interface and $interface_output are sent, $status is returned. $interface_up = handle_interface_POST_and_status($interface, $interface_output, $status); ?> - -
    -
    -
    -
    WLAN Dashboard
    isMessage()) echo "

    " . $status->showMessages() . "

    "; ?>
    -

    Interface Information

    +

    Interface Information

    +
    IP Address

    Subnet Mask

    Mac Address


    @@ -113,12 +146,8 @@ function DisplayDashboard_WLAN($interface) {
    -
    - -
    -
    -
    + diff --git a/html/includes/editor.php b/html/includes/editor.php index 794752f53..ec78d90af 100644 --- a/html/includes/editor.php +++ b/html/includes/editor.php @@ -3,15 +3,13 @@ function DisplayEditor() { $status = new StatusMessages(); - $showFullList = false; // show the full list of what's in ALLSKY_SCRIPTS, or just user-editable files? - $config_dir = basename(ALLSKY_CONFIG); ?> + + + + // The remaining code is to keep the "Save changes" button at the top of the "settings" page. + // Get the button: + let mybutton = document.getElementById("backToTopBtn"); + + if (mybutton !== null) { + // When the user scrolls down 20px from the top of the document, show the button + window.onscroll = function() {scrollFunction()}; + + function scrollFunction() { + if (document.body.scrollTop > 20 || document.documentElement.scrollTop > 20) { + mybutton.style.display = "block"; + } else { + mybutton.style.display = "none"; + } + } + + // When the user clicks on the button, scroll to the top of the document + function topFunction() { + document.body.scrollTop = 0; // For Safari + document.documentElement.scrollTop = 0; // For Chrome, Firefox, IE and Opera + } + } + + diff --git a/html/js/overlay/oe-uimanager.js b/html/js/overlay/oe-uimanager.js index a1b08747b..cfc3d0052 100644 --- a/html/js/overlay/oe-uimanager.js +++ b/html/js/overlay/oe-uimanager.js @@ -126,6 +126,10 @@ class OEUIMANAGER { this.#testMode = state; } + get editorStage() { + return this.#oeEditorStage; + } + get transformer() { return this.#transformer; } @@ -712,7 +716,7 @@ class OEUIMANAGER { field.source = fieldSource; } } else { - let fieldId = this.#configManager.dataFields.length; + let fieldId = this.#configManager.dataFields.length + 1; let newField = { id: fieldId, name: fieldName, @@ -1118,20 +1122,24 @@ class OEUIMANAGER { } setTransformerState(shape) { - if (this.#transformer.borderStroke() !== '#00a1ff') { - this.#transformer.borderStroke('#00a1ff'); - this.#transformer.borderStrokeWidth(1); + this.checkImageBounds(shape, this.#oeEditorStage, this.#transformer); + } + + checkImageBounds(shape, oeEditorStage, transformer) { + if (transformer.borderStroke() !== '#00a1ff') { + transformer.borderStroke('#00a1ff'); + transformer.borderStrokeWidth(1); } - if (shape.getClassName() == 'Image') { - let stageWidth = this.#oeEditorStage.width(); - let stageHeight = this.#oeEditorStage.height(); + if (shape.getClassName() == 'Image') { + let stageWidth = oeEditorStage.width(); + let stageHeight = oeEditorStage.height(); let rect = shape.getClientRect(); - let x = rect.x / this.#oeEditorStage.scaleX(); - let y = rect.y / this.#oeEditorStage.scaleY(); - let width = rect.width / this.#oeEditorStage.scaleX(); - let height = rect.height / this.#oeEditorStage.scaleY(); + let x = rect.x / oeEditorStage.scaleX(); + let y = rect.y / oeEditorStage.scaleY(); + let width = rect.width / oeEditorStage.scaleX(); + let height = rect.height / oeEditorStage.scaleY(); let outOfBounds = false; if (x < 0) { @@ -1149,8 +1157,8 @@ class OEUIMANAGER { } if (outOfBounds) { - this.#transformer.borderStrokeWidth(3); - this.#transformer.borderStroke('red'); + transformer.borderStrokeWidth(3); + transformer.borderStroke('red'); } } } @@ -1680,8 +1688,8 @@ class OEUIMANAGER { sample: { group: 'Label', name: 'Sample', type: 'text' }, empty: { group: 'Label', name: 'Empty Value', type: 'text' }, - x: { group: 'Position', name: 'X', type: 'number', options: { min: 0, max: 2048, step: gridSizeX } }, - y: { group: 'Position', name: 'Y', type: 'number', options: { min: 0, max: 2048, step: gridSizeY } }, + x: { group: 'Position', name: 'X', type: 'number', options: { min: 0, max: this.#backgroundImage.width(), step: gridSizeX } }, + y: { group: 'Position', name: 'Y', type: 'number', options: { min: 0, max: this.#backgroundImage.height(), step: gridSizeY } }, rotation: { group: 'Position', name: 'Rotation', type: 'number', options: { min: -360, max: 360, step: 1 } }, fontname: { group: 'Font', name: 'Name', type: 'options', options: this.#fonts }, @@ -1789,8 +1797,8 @@ class OEUIMANAGER { //image: { group: 'Image', name: 'Image', type: 'text' }, //fontname: { group: 'Font', name: 'Name', type: 'options', options: this.#fonts }, - x: { group: 'Position', name: 'X', type: 'number', options: { min: 0, max: 2048, step: 10 } }, - y: { group: 'Position', name: 'Y', type: 'number', options: { min: 0, max: 2048, step: 10 } }, + x: { group: 'Position', name: 'X', type: 'number', options: { min: 0, max: this.#backgroundImage.width(), step: 10 } }, + y: { group: 'Position', name: 'Y', type: 'number', options: { min: 0, max: this.#backgroundImage.height(), step: 10 } }, rotation: { group: 'Position', name: 'Rotation', type: 'number', options: { min: -360, max: 360, step: 1 } }, opacity: { group: 'Position', name: 'Opacity', type: 'number', options: { min: 0, max: 1, step: 0.1 } }, @@ -1806,6 +1814,7 @@ class OEUIMANAGER { // TODO: Check setter actually exists if (name !== 'image') { field[name] = value; + uiManager.checkImageBounds(field.shape, uiManager.editorStage , uiManager.transformer); } else { field.setImage(value).then( () => { uiManager.transformer.forceUpdate(); diff --git a/install.sh b/install.sh index db7551617..98f687780 100755 --- a/install.sh +++ b/install.sh @@ -1,4 +1,5 @@ #!/bin/bash +# shellcheck disable=SC2154 # referenced but not assigned [[ -z ${ALLSKY_HOME} ]] && export ALLSKY_HOME="$(realpath "$(dirname "${BASH_ARGV0}")")" ME="$(basename "${BASH_ARGV0}")" @@ -31,7 +32,7 @@ TITLE="Allsky Installer" FINAL_SUDOERS_FILE="/etc/sudoers.d/allsky" OLD_RASPAP_DIR="/etc/raspap" # used to contain WebUI configuration files SETTINGS_FILE_NAME="$(basename "${SETTINGS_FILE}")" -FORCE_CREATING_SETTINGS_FILE="false" # should a default settings file be created? +FORCE_CREATING_DEFAULT_SETTINGS_FILE="false" # should a default settings file be created? RESTORED_PRIOR_SETTINGS_FILE="false" PRIOR_SETTINGS_FILE="" # Full pathname to the prior settings file, if it exists RESTORED_PRIOR_CONFIG_SH="false" # prior config.sh restored? @@ -59,13 +60,32 @@ rm -f "${ALLSKY_MESSAGES}" # Start out with no messages. # shellcheck disable=SC2034 DISPLAY_MSG_LOG="${ALLSKY_INSTALLATION_LOGS}/install.sh.log" +# Is a reboot needed at end of installation? +REBOOT_NEEDED="true" + +# Holds status of installation if we need to exit and get back in. +STATUS_FILE="${ALLSKY_INSTALLATION_LOGS}/status.txt" +STATUS_FILE_TEMP="${ALLSKY_TMP}/temp_status.txt" # holds intermediate status +STATUS_LOCALE_REBOOT="Rebooting to change locale" # status of rebooting due to locale change +STATUS_FINISH_REBOOT="Rebooting to finish installation" +STATUS_NO_FINISH_REBOOT="Did not reboot to finish installation" +STATUS_NO_REBOOT="User elected not to reboot" +STATUS_NO_LOCALE="Desired locale not found" # exiting due to desired locale not installed +STATUS_NO_CAMERA="No camera found" # status of exiting due to no camera found +STATUS_OK="OK" # Installation was completed. +STATUS_NOT_CONTINUE="User elected not to continue" # Exiting, but not an error +STATUS_CLEAR="Clear" # Clear the file +STATUS_ERROR="Error encountered" +STATUS_INT="Got interrupt" +STATUS_VARIABLES=() # Holds all the variables and values to save # Some versions of Linux default to 750 so web server can't read it chmod 755 "${ALLSKY_HOME}" +OS="$(grep CODENAME /etc/os-release | cut -d= -f2)" # usually buster or bullseye -####################### functions +############################################## functions #### # @@ -76,31 +96,38 @@ do_initial_heading() return fi - MSG="Welcome to the ${TITLE}!\n" + if [[ ${do_initial_heading} == "true" ]]; then + display_header "Welcome back to the ${TITLE}!" + else + MSG="Welcome to the ${TITLE}!\n" - if [[ -n ${PRIOR_ALLSKY} ]]; then - MSG="${MSG}\nYou will be asked if you want to use the images and darks (if any) from" - MSG="${MSG}\nyour prior version of Allsky." - if [[ ${PRIOR_ALLSKY} == "new" ]]; then - MSG="${MSG}\nIf so, we will use its settings as well." + if [[ -n ${PRIOR_ALLSKY} ]]; then + MSG="${MSG}\nYou will be asked if you want to use the images and darks (if any) from" + MSG="${MSG}\nyour prior version of Allsky." + if [[ ${PRIOR_ALLSKY} == "newStyle" ]]; then + MSG="${MSG}\nIf so, its settings will be used as well." + else + MSG="${MSG}\nIf so, we will attempt to use its settings as well, but may not be" + MSG="${MSG}\nable to use ALL prior settings depending on how old your prior Allsky is." + MSG="${MSG}\nIn that case, you'll be prompted for required information such as" + MSG="${MSG}\nthe camera's latitude, logitude, and locale." + fi else - MSG="${MSG}\nIf so, we will attempt to use its settings as well, but may not be" - MSG="${MSG}\nable to use ALL prior settings depending on how old your prior Allsky is." - MSG="${MSG}\nIn that case, you'll be prompted for required information such as" - MSG="${MSG}\nthe camera's latitude, logitude, and locale." + MSG="${MSG}\nYou will be prompted for required information such as the type" + MSG="${MSG}\nof camera you have and the camera's latitude, logitude, and locale." fi - else - MSG="${MSG}\nYou will be prompted for required information such as the type" - MSG="${MSG}\nof camera you have and the camera's latitude, logitude, and locale." - fi - MSG="${MSG}\n\nNOTE: your camera must be connected to the Pi before continuing." - MSG="${MSG}\n\nContinue?" - if ! whiptail --title "${TITLE}" --yesno "${MSG}" 25 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then - display_msg "${LOG_TYPE}" info "User not ready to continue." - exit_installation 1 + MSG="${MSG}\n\nNOTE: your camera must be connected to the Pi before continuing." + MSG="${MSG}\n\nContinue?" + if ! whiptail --title "${TITLE}" --yesno "${MSG}" 25 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + display_msg "${LOG_TYPE}" info "User not ready to continue." + exit_installation 1 "${STATUS_CLEAR}" "" + fi + + display_header "Welcome to the ${TITLE}!" fi - display_header "Welcome to the ${TITLE}!" + + [[ ${do_initial_heading} != "true" ]] && STATUS_VARIABLES+=("do_initial_heading='true'\n") } #### @@ -139,28 +166,17 @@ stop_allsky() #### # Get the branch of the release we are installing; -# if not GITHUB_MAIN_BRANCH, save the name of the branch. -# There is no "branch" file in GitHub so we need to determine the branch -# by looking in the .git/config file. get_this_branch() { - local FILE="${ALLSKY_HOME}/.git/config" - if [[ -f ${FILE} ]]; then - local B="$(sed -E --silent -e '/^\[branch "/s/(^\[branch ")(.*)("])/\2/p' "${FILE}")" - if [[ -n ${B} ]]; then - if [[ ${B} == "${GITHUB_MAIN_BRANCH}" ]]; then - display_msg --logonly info "Using the '${B}' branch." - else - BRANCH="${B}" - echo -n "${BRANCH}" > "${ALLSKY_BRANCH_FILE}" - display_msg --log info "Using '${BRANCH}' branch." - fi - else - display_msg --log warning "Unable to determine branch from '${FILE}'; assuming ${BRANCH}." - fi + if ! B="$( get_branch )" ; then + display_msg --log warning "Unable to determine branch; assuming '${BRANCH}'." else - display_msg --log warning "${FILE} not found; assuming ${BRANCH} branch." + BRANCH="${B}" + display_msg --logonly info "Using the '${BRANCH}' branch." fi + + STATUS_VARIABLES+=("get_this_branch='true'\n") + STATUS_VARIABLES+=("BRANCH='${BRANCH}'\n") } @@ -191,7 +207,7 @@ CAMERA_TYPE_to_CAMERA() echo "RPiHQ" # RPi cameras used to be called "RPiHQ". else display_msg --log error "Unknown CAMERA_TYPE: '${CAMERA_TYPE}'" - exit_installation 1 + exit_installation 1 "${STATUS_ERROR}" "unknown CAMERA_TYPE: '${CAMERA_TYPE}'" fi } #### @@ -205,38 +221,87 @@ CAMERA_to_CAMERA_TYPE() echo "RPi" else display_msg --log error "Unknown CAMERA: '${CAMERA}'" - exit_installation 1 + exit_installation 1 "${STATUS_CLEAR}" "unknown CAMERA: '${CAMERA}'" fi } -#### +####### +CONNECTED_CAMERAS="" +get_connected_cameras() +{ + local CC + if determineCommandToUse "false" "" > /dev/null 2>&1 ; then + display_msg --log info "RPi camera found." + CC="RPi" + fi + if lsusb -d "03c3:" > /dev/null ; then + display_msg --log info "ZWO camera found." + [[ -n ${CC} ]] && CC="${CC} " + CC="${CC}ZWO" + fi + + if [[ -z ${CC} ]]; then + MSG="No connected cameras were detected. The installation will exit." + whiptail --title "${TITLE}" --msgbox "${MSG}" 12 "${WT_WIDTH}" 3>&1 1>&2 2>&3 + + MSG="No connected cameras were detected." + MSG="${MSG}\nMake sure a camera is plugged in and working prior to restarting" + MSG="${MSG} the installation." + display_msg --log error "${MSG}" + exit_installation 1 "${STATUS_NO_CAMERA}" "" + fi + + if [[ -n ${CONNECTED_CAMERAS} ]]; then + # Set from a prior installation. + if [[ ${CONNECTED_CAMERAS} != "${CC}" ]]; then + MSG="Connected cameras were '${CONNECTED_CAMERAS}' during last installation" + MSG="${MSG} but are '${CC}' now." + display_msg --log info "${MSG}" + STATUS_VARIABLES+=("CONNECTED_CAMERAS='${CC}'\n") + fi + # Else the last one and this one are the same so don't save. + CONNECTED_CAMERAS="${CC}" + return + fi + + [[ ${get_connected_cameras} != "true" ]] && STATUS_VARIABLES+=("get_connected_cameras='true'\n") + # Either not set before or is different this time + CONNECTED_CAMERAS="${CC}" +} + +# # Prompt the user to select their camera type, if we can't determine it automatically. # If they have a prior installation of Allsky that uses either CAMERA or CAMERA_TYPE in config.sh, # we can use its value and not prompt. CAMERA_TYPE="" select_camera_type() { - if [[ -f ${PRIOR_CONFIG_FILE} ]]; then + if [[ -n ${PRIOR_ALLSKY} ]]; then case "${PRIOR_ALLSKY_VERSION}" in # New versions go here... v2023.05.01*) # New style Allsky using ${CAMERA_TYPE}. - CAMERA_TYPE="$(get_variable "CAMERA_TYPE" "${PRIOR_CONFIG_FILE}")" + CAMERA_TYPE="${PRIOR_CAMERA_TYPE}" + # Don't bother with a message since this is a "similar" release. if [[ -n ${CAMERA_TYPE} ]]; then - MSG="Using CAMERA_TYPE '${CAMERA_TYPE}' from prior config.sh" + MSG="Using Camera Type '${CAMERA_TYPE}' from prior Allsky." + STATUS_VARIABLES+=("select_camera_type='true'\n") + STATUS_VARIABLES+=("CAMERA_TYPE='${CAMERA_TYPE}'\n") display_msg --logonly info "${MSG}" return else - MSG="CAMERA_TYPE not in prior config.sh; possibly corrupted file?" + MSG="Camera Type not in prior new-style settings file." display_msg --log error "${MSG}" fi ;; "v2022.03.01" | "old") - local CAMERA="$(get_variable "CAMERA" "${PRIOR_CONFIG_FILE}")" + local CAMERA="$( get_variable "CAMERA" "${PRIOR_CONFIG_FILE}" )" if [[ -n ${CAMERA} ]]; then CAMERA_TYPE="$( CAMERA_to_CAMERA_TYPE "${CAMERA}" )" + STATUS_VARIABLES+=("select_camera_type='true'\n") + STATUS_VARIABLES+=("CAMERA_TYPE='${CAMERA_TYPE}'\n") if [[ ${CAMERA} != "${CAMERA_TYPE}" ]]; then NEW=" (now called ${CAMERA_TYPE})" else @@ -252,17 +317,46 @@ select_camera_type() esac fi - # "2" is the number of menu items. - MSG="\nSelect your camera type:\n" - CAMERA_TYPE=$(whiptail --title "${TITLE}" --menu "${MSG}" 15 "${WT_WIDTH}" 2 \ - "ZWO" " ZWO ASI" \ - "RPi" " Raspberry Pi HQ, Module 3, and compatible" \ - 3>&1 1>&2 2>&3) + local CT=() + local NUM=0 + if [[ ${CONNECTED_CAMERAS} == "RPi" ]]; then + CT+=("RPi" " Raspberry Pi (HQ, Module 3, and compatibles)") + ((NUM++)) + elif [[ ${CONNECTED_CAMERAS} == "ZWO" ]]; then + CT+=("ZWO" " ZWO ASI") + ((NUM++)) + elif [[ ${CONNECTED_CAMERAS} == "RPi ZWO" ]]; then + CT+=("RPi" " Raspberry Pi (HQ, Module 3, and compatibles)") + CT+=("ZWO" " ZWO ASI") + ((NUM+=2)) + else # shouldn't happen since we already checked + MSG="INTERNAL ERROR:" + if [[ -z ${CONNECTED_CAMERAS} ]]; then + MSG="${MSG} CONNECTED_CAMERAS is empty." + else + MSG="${MSG} CONNECTED_CAMERAS (${CONNECTED_CAMERAS}) is invalid." + fi + display_msg --log error "${MSG}" + exit_installation 2 "${STATUS_NO_CAMERA}" "${MSG}" + fi + + local S=" is" + [[ ${NUM} -gt 1 ]] && S="s are" + MSG="\nThe following camera type${S} connected to the Pi.\n" + MSG="${MSG}Pick the one you want." + MSG="${MSG}\nIf it's not in the list, select and determine why." + CAMERA_TYPE=$(whiptail --title "${TITLE}" --menu "${MSG}" 15 "${WT_WIDTH}" "${NUM}" \ + "${CT[@]}" 3>&1 1>&2 2>&3) if [[ $? -ne 0 ]]; then - display_msg --log warning "Camera selection required. Please re-run the installation and select a camera to continue." - exit_installation 2 + MSG="Camera selection required." + MSG="${MSG} Please re-run the installation and select a camera to continue." + display_msg --log warning "${MSG}" + exit_installation 2 "${STATUS_NO_CAMERA}" "User did not select a camera." fi + display_msg --log progress "Using ${CAMERA_TYPE} camera." + STATUS_VARIABLES+=("select_camera_type='true'\n") + STATUS_VARIABLES+=("CAMERA_TYPE='${CAMERA_TYPE}'\n") } @@ -294,6 +388,8 @@ create_webui_defines() -e "s;XX_ALLSKY_MODULES_XX;${ALLSKY_MODULES};" \ "${REPO_WEBUI_DEFINES_FILE}" > "${FILE}" chmod 644 "${FILE}" + + STATUS_VARIABLES+=("create_webui_defines='true'\n") } @@ -302,7 +398,7 @@ create_webui_defines() # This can be used after installation if the options file gets hosed. recreate_options_file() { - CAMERA_TYPE="$(get_variable "CAMERA_TYPE" "${ALLSKY_CONFIG}/config.sh")" + CAMERA_TYPE="$( get_variable "CAMERA_TYPE" "${ALLSKY_CONFIG}/config.sh" )" save_camera_capabilities "true" set_permissions } @@ -321,12 +417,13 @@ save_camera_capabilities() return 1 fi - OPTIONSFILEONLY="${1}" # Set to "true" if we should ONLY create the options file. + local OPTIONSFILEONLY="${1}" # Set to "true" if we should ONLY create the options file. + local FORCE MSG OPTIONSONLY # Create the camera type/model-specific options file and optionally a default settings file. # --cameraTypeOnly tells makeChanges.sh to only change the camera info, then exit. # It displays any error messages. - if [[ ${FORCE_CREATING_SETTINGS_FILE} == "true" ]]; then + if [[ ${FORCE_CREATING_DEFAULT_SETTINGS_FILE} == "true" ]]; then FORCE="--force" MSG=" and default settings" else @@ -341,35 +438,41 @@ save_camera_capabilities() display_msg --log progress "Setting up WebUI options${MSG} for ${CAMERA_TYPE} cameras." fi - # Restore the prior settings file so it can be used by makeChanges.sh. - [[ ${PRIOR_ALLSKY} != "" ]] && restore_prior_settings_files + # Restore the prior settings file or camera-specific settings file(s) so + # the appropriate one can be used by makeChanges.sh. + [[ ${PRIOR_ALLSKY} != "" ]] && restore_prior_settings_file + + display_msg --log progress "Making new settings file '${SETTINGS_FILE}'." MSG="Executing makeChanges.sh ${FORCE} ${OPTIONSONLY} --cameraTypeOnly" MSG="${MSG} ${DEBUG_ARG} 'cameraType' 'Camera Type' '${PRIOR_CAMERA_TYPE}' '${CAMERA_TYPE}'" display_msg "${LOG_TYPE}" info "${MSG}" #shellcheck disable=SC2086 - MSG="$( "${ALLSKY_SCRIPTS}/makeChanges.sh" ${FORCE} ${OPTIONSONLY} --cameraTypeOnly ${DEBUG_ARG} \ - "cameraType" "Camera Type" "${PRIOR_CAMERA_TYPE}" "${CAMERA_TYPE}" 2>&1 )" + MSG="$( "${ALLSKY_SCRIPTS}/makeChanges.sh" ${FORCE} ${OPTIONSONLY} --cameraTypeOnly \ + ${DEBUG_ARG} "cameraType" "Camera Type" "${PRIOR_CAMERA_TYPE}" "${CAMERA_TYPE}" 2>&1 )" RET=$? - display_msg "${LOG_TYPE}" info "${MSG}" + [[ -n ${MSG} ]] && display_msg "${LOG_TYPE}" info "${MSG}" if [[ ${RET} -ne 0 ]]; then #shellcheck disable=SC2086 if [[ ${RET} -eq ${EXIT_NO_CAMERA} ]]; then MSG="No camera was found; one must be connected and working for the installation to succeed.\n" - MSG="$MSG}After connecting your camera, run '${ME} --update'." + MSG="$MSG}After connecting your camera, re-run the installation." whiptail --title "${TITLE}" --msgbox "${MSG}" 12 "${WT_WIDTH}" 3>&1 1>&2 2>&3 display_msg --log error "No camera detected - installation aborted." elif [[ ${OPTIONSFILEONLY} == "false" ]]; then display_msg --log error "Unable to save camera capabilities." fi return 1 - else - MSG="$( ls -l "${ALLSKY_CONFIG}/settings"*.json 2>/dev/null )" - display_msg "${LOG_TYPE}" info "Settings files:\n${MSG}" fi + #shellcheck disable=SC2012 + MSG="$( /bin/ls -l "${ALLSKY_CONFIG}/settings"*.json 2>/dev/null | sed 's/^/ /' )" + display_msg "${LOG_TYPE}" info "Settings files:\n${MSG}" + CAMERA_MODEL="$( settings ".cameraModel" "${SETTINGS_FILE}" )" + + STATUS_VARIABLES+=("save_camera_capabilities='true'\n") return 0 } @@ -414,17 +517,17 @@ ask_reboot() WILL_REBOOT="true" display_msg --logonly info "Pi will reboot after installation completes." else - display_msg --logonly info "User elected not to reboot; warning to them provided." + display_msg --logonly info "User elected not to reboot; displayed warning message." display_msg notice "You need to reboot the Pi before Allsky will work." MSG="If you have not already rebooted your Pi, please do so now.\n" MSG="${MSG}You can then connect to the WebUI at:\n" MSG="${MSG}${AT}" - echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" + "${ALLSKY_SCRIPTS}/addMessage.sh" "info" "${MSG}" fi } do_reboot() { - exit_installation -1 # -1 means just log ending statement but don't exit. + exit_installation -1 "${1}" "${2}" # -1 means just log ending statement but don't exit. sudo reboot now } @@ -439,22 +542,21 @@ recheck_swap() } check_swap() { - if [[ ${1} == "prompt" ]]; then - PROMPT="true" - else - PROMPT="false" - fi + STATUS_VARIABLES+=("check_swap='true'\n") + + local PROMPT="false" + [[ ${1} == "prompt" ]] && PROMPT="true" # This can return "total_mem is unknown" if the OS is REALLY old. - RAM_SIZE="$( vcgencmd get_config total_mem )" + local RAM_SIZE="$( vcgencmd get_config total_mem )" if echo "${RAM_SIZE}" | grep --silent "unknown" ; then # Note: This doesn't produce exact results. On a 4 GB Pi, it returns 3.74805. RAM_SIZE=$(free --mebi | awk '{if ($1 == "Mem:") {print $2; exit 0} }') # in MB else RAM_SIZE="${RAM_SIZE//total_mem=/}" fi - DESIRED_COMBINATION=$((1024 * 4)) # desired minimum memory + swap - SUGGESTED_SWAP_SIZE=0 + local DESIRED_COMBINATION=$((1024 * 4)) # desired minimum memory + swap + local SUGGESTED_SWAP_SIZE=0 for i in 512 1024 2048 # 4096 and above don't need any swap do if [[ ${RAM_SIZE} -le ${i} ]]; then @@ -462,15 +564,16 @@ check_swap() break fi done - display_msg "${LOG_TYPE}" debug "RAM_SIZE=${RAM_SIZE}, SUGGESTED_SWAP_SIZE=${SUGGESTED_SWAP_SIZE}." + display_msg --logonly info "RAM_SIZE=${RAM_SIZE}, SUGGESTED_SWAP_SIZE=${SUGGESTED_SWAP_SIZE}." # Not sure why, but displayed swap is often 1 MB less than what's in /etc/dphys-swapfile - CURRENT_SWAP=$(free --mebi | awk '{if ($1 == "Swap:") {print $2 + 1; exit 0} }') # in MB + local CURRENT_SWAP=$(free --mebi | awk '{if ($1 == "Swap:") {print $2 + 1; exit 0} }') # in MB CURRENT_SWAP=${CURRENT_SWAP:-0} if [[ ${CURRENT_SWAP} -lt ${SUGGESTED_SWAP_SIZE} || ${PROMPT} == "true" ]]; then local SWAP_CONFIG_FILE="/etc/dphys-swapfile" [[ -z ${FUNCTION} ]] && sleep 2 # give user time to read prior messages + local AMT M if [[ ${CURRENT_SWAP} -eq 1 ]]; then CURRENT_SWAP=0 AMT="no" @@ -491,7 +594,7 @@ check_swap() MSG="${MSG}\n\nYou may change the amount of swap by changing the number below." fi - SWAP_SIZE=$(whiptail --title "${TITLE}" --inputbox "${MSG}" 18 "${WT_WIDTH}" \ + local SWAP_SIZE=$(whiptail --title "${TITLE}" --inputbox "${MSG}" 18 "${WT_WIDTH}" \ "${SUGGESTED_SWAP_SIZE}" 3>&1 1>&2 2>&3) # If the suggested swap was 0 and the user added a number but didn't first delete the 0, # do it now so we don't have numbers like "0256". @@ -504,11 +607,11 @@ check_swap() display_msg --log info "Swap will remain at ${CURRENT_SWAP}." fi else - display_msg --log progress "Setting swap space set to ${SWAP_SIZE} MB." + display_msg --log progress "Setting swap space to ${SWAP_SIZE} MB." sudo dphys-swapfile swapoff # Stops the swap file sudo sed -i "/CONF_SWAPSIZE/ c CONF_SWAPSIZE=${SWAP_SIZE}" "${SWAP_CONFIG_FILE}" - CURRENT_MAX="$(get_variable "CONF_MAXSWAP" "${SWAP_CONFIG_FILE}")" + local CURRENT_MAX="$(get_variable "CONF_MAXSWAP" "${SWAP_CONFIG_FILE}")" # TODO: Can we determine the default max rather than hard-code it. CURRENT_MAX="${CURRENT_MAX:-2048}" if [[ ${CURRENT_MAX} -lt ${SWAP_SIZE} ]]; then @@ -537,7 +640,6 @@ check_and_mount_tmp() if [[ -d "${ALLSKY_TMP}" ]]; then local IMAGES="$(find "${ALLSKY_TMP}" -name '*.jpg')" - display_msg "${LOG_TYPE}" debug "Existing IMAGES=${IMAGES}" if [[ -n ${IMAGES} ]]; then mkdir "${TMP_DIR}" # Need to allow for files with spaces in their names. @@ -567,16 +669,20 @@ check_and_mount_tmp() # If not, offer to make it one. check_tmp() { - INITIAL_FSTAB_STRING="tmpfs ${ALLSKY_TMP} tmpfs" + STATUS_VARIABLES+=("check_tmp='true'\n") + + local INITIAL_FSTAB_STRING="tmpfs ${ALLSKY_TMP} tmpfs" # Check if currently a memory filesystem. if grep --quiet "^${INITIAL_FSTAB_STRING}" /etc/fstab; then - display_msg --log progress "${ALLSKY_TMP} is currently in memory; no change needed." + MSG="${ALLSKY_TMP} is currently a memory filesystem; no change needed." + display_msg --log progress "${MSG}" # If there's a prior Allsky version and it's tmp directory is mounted, # try to unmount it, but that often gives an error that it's busy, # which isn't really a problem since it'll be unmounted at the reboot. - # /etc/fstab has ${ALLSKY_TMP} but the mount point is currently in the PRIOR Allsky. + # We know from the grep above that /etc/fstab has ${ALLSKY_TMP} + # but the mount point is currently in the PRIOR Allsky. local D="${PRIOR_ALLSKY_DIR}/tmp" if [[ -d "${D}" ]] && mount | grep --silent "${D}" ; then # The Samba daemon is one known cause of "target busy". @@ -590,7 +696,7 @@ check_tmp() # If the new Allsky's ${ALLSKY_TMP} is already mounted, don't do anything. # This would be the case during an upgrade. if mount | grep --silent "${ALLSKY_TMP}" ; then - display_msg "${LOG_TYPE}" debug "${ALLSKY_TMP} already mounted" + display_msg --logonly info "${ALLSKY_TMP} already mounted." return 0 fi @@ -598,7 +704,7 @@ check_tmp() return 0 fi - SIZE=75 # MB - should be enough + local SIZE=75 # MB - should be enough MSG="Making ${ALLSKY_TMP} reside in memory can drastically decrease the amount of writes to the SD card, increasing its life." MSG="${MSG}\n\nDo you want to make it reside in memory?" MSG="${MSG}\n\nNote: anything in it will be deleted whenever the Pi is rebooted, but that's not an issue since the directory only contains temporary files." @@ -648,17 +754,19 @@ install_webserver() sudo apt-get update && \ sudo apt-get --assume-yes install lighttpd php-cgi php-gd hostapd dnsmasq avahi-daemon ) > "${TMP}" 2>&1 - check_success $? "lighttpd installation failed" "${TMP}" "${DEBUG}" || exit_with_image 1 + if ! check_success $? "lighttpd installation failed" "${TMP}" "${DEBUG}" ; then + exit_with_image 1 "${STATUS_ERROR}" "lighttpd installation failed" + fi FINAL_LIGHTTPD_FILE="/etc/lighttpd/lighttpd.conf" sed \ -e "s;XX_ALLSKY_WEBUI_XX;${ALLSKY_WEBUI};g" \ -e "s;XX_ALLSKY_HOME_XX;${ALLSKY_HOME};g" \ -e "s;XX_ALLSKY_IMAGES_XX;${ALLSKY_IMAGES};g" \ - -e "s;XX_ALLSKY_WEBSITE_XX;${ALLSKY_WEBSITE};g" \ - -e "s;XX_ALLSKY_DOCUMENTATION_XX;${ALLSKY_DOCUMENTATION};g" \ -e "s;XX_ALLSKY_CONFIG_XX;${ALLSKY_CONFIG};g" \ + -e "s;XX_ALLSKY_WEBSITE_XX;${ALLSKY_WEBSITE};g" \ -e "s;XX_ALLSKY_OVERLAY_XX;${ALLSKY_OVERLAY};g" \ + -e "s;XX_ALLSKY_DOCUMENTATION_XX;${ALLSKY_DOCUMENTATION};g" \ "${REPO_LIGHTTPD_FILE}" > /tmp/x sudo install -m 0644 /tmp/x "${FINAL_LIGHTTPD_FILE}" && rm -f /tmp/x @@ -678,42 +786,50 @@ install_webserver() sudo systemctl start lighttpd # Starting it added an entry so truncate the file so it's 0-length sleep 1; truncate -s 0 "${LIGHTTPD_LOG}" + + STATUS_VARIABLES+=("install_webserver='true'\n") } #### # Prompt for a new hostname if needed, # and update all the files that contain the hostname. +# The default hostname in Pi OS is "raspberrypi"; if it's still that, +# prompt to update. If it's anything else that means the user +# already changed it to something so don't overwrite their change. + prompt_for_hostname() { - # If the Pi is already called ${SUGGESTED_NEW_HOST_NAME}, - # then the user already updated the name, so don't prompt again. - - CURRENT_HOSTNAME=$(tr -d " \t\n\r" < /etc/hostname) - if [[ ${CURRENT_HOSTNAME} == "${SUGGESTED_NEW_HOST_NAME}" ]]; then + local CURRENT_HOSTNAME=$(tr -d " \t\n\r" < /etc/hostname) + if [[ ${CURRENT_HOSTNAME} != "raspberrypi" ]]; then + display_msg --logonly info "Using current hostname of '${CURRENT_HOSTNAME}'." NEW_HOST_NAME="${CURRENT_HOSTNAME}" - return - fi - # If we're upgrading, use the current name. - if [[ -n ${PRIOR_ALLSKY} ]]; then - NEW_HOST_NAME="${CURRENT_HOSTNAME}" - display_msg --log progress "Using current hostname of ${CURRENT_HOSTNAME}." + STATUS_VARIABLES+=("prompt_for_hostname='true'\n") + STATUS_VARIABLES+=("NEW_HOST_NAME='${NEW_HOST_NAME}'\n") return fi MSG="Please enter a hostname for your Pi." - MSG="${MSG}\n\nIf you have more than one Pi on your network they must all have unique names." - NEW_HOST_NAME=$(whiptail --title "${TITLE}" --inputbox "${MSG}" 10 "${WT_WIDTH}" \ + MSG="${MSG}\n\nIf you have more than one Pi on your network they MUST all have unique names." + MSG="${MSG}\n\nThe current hostname is '${CURRENT_HOSTNAME}'; the suggested name is below:\n" + NEW_HOST_NAME=$(whiptail --title "${TITLE}" --inputbox "${MSG}" 15 "${WT_WIDTH}" \ "${SUGGESTED_NEW_HOST_NAME}" 3>&1 1>&2 2>&3) if [[ $? -ne 0 ]]; then - display_msg --log warning "You must specify a host name. Please re-run the installation and select one." - exit_installation 2 + MSG="You must specify a host name." + MSG="${MSG} Please re-run the installation and select one." + display_msg --log warning "${MSG}" + exit_installation 2 "No host name selected" + else + STATUS_VARIABLES+=("prompt_for_hostname='true'\n") + STATUS_VARIABLES+=("NEW_HOST_NAME='${NEW_HOST_NAME}'\n") fi if [[ ${CURRENT_HOSTNAME} != "${NEW_HOST_NAME}" ]]; then echo "${NEW_HOST_NAME}" | sudo tee /etc/hostname > /dev/null sudo sed -i "s/127.0.1.1.*${CURRENT_HOSTNAME}/127.0.1.1\t${NEW_HOST_NAME}/" /etc/hosts + + # else, they didn't change the default name, but that's their problem... fi # Set up the avahi daemon if needed. @@ -767,16 +883,20 @@ set_permissions() # The web server needs to be able to create and update many of the files in ${ALLSKY_CONFIG}. # Not all, but go ahead and chgrp all of them so we don't miss any new ones. - sudo find "${ALLSKY_CONFIG}/" -type f -exec chmod 664 {} \; - sudo find "${ALLSKY_CONFIG}/" -type d -exec chmod 775 {} \; + sudo find "${ALLSKY_CONFIG}/" -type f -exec chmod 664 '{}' \; + sudo find "${ALLSKY_CONFIG}/" -type d -exec chmod 775 '{}' \; sudo chgrp -R "${WEBSERVER_GROUP}" "${ALLSKY_CONFIG}" # The files should already be the correct permissions/owners, but just in case, set them. # We don't know what permissions may have been on the old website, so use "sudo". - sudo find "${ALLSKY_WEBUI}/" -type f -exec chmod 644 {} \; - # These are the exceptions + sudo find "${ALLSKY_WEBUI}/" -type f -exec chmod 644 '{}' \; + sudo find "${ALLSKY_WEBUI}/" -type d -exec chmod 755 '{}' \; chmod 755 "${ALLSKY_WEBUI}/includes/createAllskyOptions.php" - sudo find "${ALLSKY_WEBUI}/" -type d -exec chmod 755 {} \; + + if [[ -d "${ALLSKY_WEBSITE}" ]]; then + sudo find "${ALLSKY_WEBUI}/" -type d -name thumbnails \! -perm 775 -exec chmod 775 '{}' \; + sudo find "${ALLSKY_WEBUI}/" -type d -name thumbnails \! -group "${WEBSERVER_GROUP}" -exec chgrp "${WEBSERVER_GROUP}" '{}' \; + fi chmod 775 "${ALLSKY_TMP}" sudo chgrp "${WEBSERVER_GROUP}" "${ALLSKY_TMP}" @@ -796,8 +916,17 @@ OLD_WEBUI_LOCATION_EXISTS_AT_START="false" does_old_WebUI_location_exist() { [[ -d ${OLD_WEBUI_LOCATION} ]] && OLD_WEBUI_LOCATION_EXISTS_AT_START="true" + + STATUS_VARIABLES+=("does_old_WebUI_location_exist='true'\n") + STATUS_VARIABLES+=("OLD_WEBUI_LOCATION_EXISTS_AT_START='${OLD_WEBUI_LOCATION_EXISTS_AT_START}'\n") } +# If the old WebUI location is there: +# but it wasn't when the installation started, +# that means the installation created it so remove it. +# +# Let the user know if there's an old WebUI, or something unknown there. + check_old_WebUI_location() { [[ ! -d ${OLD_WEBUI_LOCATION} ]] && return @@ -813,24 +942,28 @@ check_old_WebUI_location() sudo rm -f "${OLD_WEBUI_LOCATION}/index.lighttpd.html" if [[ ! -d ${OLD_WEBUI_LOCATION}/includes ]]; then - MSG="The old WebUI location '${OLD_WEBUI_LOCATION}' exists" - COUNT=$(find "${OLD_WEBUI_LOCATION}" | wc -l) + local COUNT=$( find "${OLD_WEBUI_LOCATION}" | wc -l ) if [[ ${COUNT} -eq 1 ]]; then - MSG="${MSG} and is empty." - MSG="${MSG}\nYou can safely delete it after installation: sudo rmdir '${OLD_WEBUI_LOCATION}'" + # This is often true after a clean install of the OS. + sudo rmdir "${OLD_WEBUI_LOCATION}" + display_msg --logonly info "Deleted empty '${OLD_WEBUI_LOCATION}'." else + MSG="The old WebUI location '${OLD_WEBUI_LOCATION}' exists" MSG="${MSG} but doesn't contain a valid WebUI." MSG="${MSG}\nPlease check it out after installation - if there's nothing you" MSG="${MSG} want in it, remove it: sudo rm -fr '${OLD_WEBUI_LOCATION}'" + whiptail --title "${TITLE}" --msgbox "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3 + display_msg --log notice "${MSG}" + + echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" fi - whiptail --title "${TITLE}" --msgbox "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3 - display_msg --log notice "${MSG}" - echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" return fi - MSG="An old version of the WebUI was found in ${OLD_WEBUI_LOCATION}; it is no longer being used so you may remove it after intallation." - MSG="${MSG}\n\nWARNING: if you have any other web sites in that directory, they will no longer be accessible via the web server." + MSG="An old version of the WebUI was found in ${OLD_WEBUI_LOCATION};" + MSG="${MSG} it is no longer being used so you may remove it after intallation." + MSG="${MSG}\n\nWARNING: if you have any other web sites in that directory," + MSG="${MSG}\n\n they will no longer be accessible via the web server." whiptail --title "${TITLE}" --msgbox "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3 display_msg --log notice "${MSG}" echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" @@ -842,6 +975,9 @@ check_old_WebUI_location() # If it's a new-style website, copy to the new Allsky release directory. handle_prior_website() { + STATUS_VARIABLES+=( "handle_prior_website='true'\n" ) + # No variables to add to STATUS_VARIABLES. + local PRIOR_SITE="" local PRIOR_STYLE="" @@ -864,7 +1000,7 @@ handle_prior_website() # when we remove the prior WebUI. if [[ -d ${ALLSKY_WEBSITE} ]]; then - # Hmmm. There's an old webite AND a new one. + # Hmmm. There's prior webite AND a new one. # Allsky doesn't ship with the website directory, so not sure how one got there... # Try to remove the new one - if it's not empty the remove will fail # so rename it. @@ -879,8 +1015,8 @@ handle_prior_website() fi fi - # Trailing "/" tells get_version and get_branch to fill in the file - # names given the directory we pass to them. + # Trailing "/" tells get_version to fill in the file + # name given the directory we pass to them. # If there's no prior website version, then there IS a newer version available. # Set ${PV} to a string to display in messages, but we'll still use ${PRIOR_VERSION} @@ -895,7 +1031,7 @@ handle_prior_website() local NEWEST_VERSION="$(get_Git_version "${GITHUB_MAIN_BRANCH}" "${GITHUB_WEBSITE_PACKAGE}")" if [[ -z ${NEWEST_VERSION} ]]; then - display_msg --log warning "Unable to determine verson of GitHub branch '${GITHUB_MAIN_BRANCH}'." + display_msg --log warning "Unable to determine version of GitHub's Website branch '${GITHUB_MAIN_BRANCH}'." fi local B="" @@ -907,32 +1043,32 @@ handle_prior_website() if [[ ${PRIOR_STYLE} == "new" ]]; then - # If get_branch returns "" the prior branch is ${GITHUB_MAIN_BRANCH}. - local PRIOR_BRANCH="$( get_branch "${PRIOR_SITE}/" )" + # If get_branch() returns "" assume prior branch is ${GITHUB_MAIN_BRANCH}. + local PRIOR_BRANCH="$( get_branch "${PRIOR_SITE}" )" + PRIOR_BRANCH="${PRIOR_BRANCH:-${GITHUB_MAIN_BRANCH}}" - display_msg --log progress "Restoring Allsky Website from ${PRIOR_SITE}." + display_msg --log progress "Restoring local Allsky Website from ${PRIOR_SITE}." sudo mv "${PRIOR_SITE}" "${ALLSKY_WEBSITE}" # Update "AllskyVersion" if needed. - V="$( settings .config.AllskyVersion "${ALLSKY_WEBSITE_CONFIGURATION_FILE}" )" - display_msg "${LOG_TYPE}" info "Prior local Website's AllskyVersion=${V}" - if [[ ${V} != "${ALLSKY_VERSION}" ]]; then - MSG="Updating AllskyVersion in local Website from '${V}' to '${ALLSKY_VERSION}'" + local V="$( settings .config.AllskyVersion "${ALLSKY_WEBSITE_CONFIGURATION_FILE}" )" + if [[ ${V} == "${ALLSKY_VERSION}" ]]; then + display_msg --logonly info "Prior local Website's AllskyVersion already at '${ALLSKY_VERSION}'" + else + MSG="Updating local Website's AllskyVersion from '${V}' to '${ALLSKY_VERSION}'" display_msg --log progress "${MSG}" - jq ".config.AllskyVersion = \"${ALLSKY_VERSION}\"" \ - "${ALLSKY_WEBSITE_CONFIGURATION_FILE}" > /tmp/x \ - && mv /tmp/x "${ALLSKY_WEBSITE_CONFIGURATION_FILE}" + update_json_file ".config.AllskyVersion" "${ALLSKY_VERSION}" \ + "${ALLSKY_WEBSITE_CONFIGURATION_FILE}" fi - - # We can only check versions if we obtained the new version. + # We can only check Website versions if we obtained the new Website version. [[ -z ${NEWEST_VERSION} ]] && return # If the old Website was using a non-production branch, # see if there's a newer version of the Website with that branch OR # a newer version with the production branch. Use whichever is newer. if [[ -n ${PRIOR_BRANCH} && ${PRIOR_BRANCH} != "${GITHUB_MAIN_BRANCH}" ]]; then - NEWEST_VERSION="$(get_Git_version "${PRIOR_BRANCH}" "${GITHUB_WEBSITE_PACKAGE}")" + NEWEST_VERSION="$( get_Git_version "${PRIOR_BRANCH}" "${GITHUB_WEBSITE_PACKAGE}" )" B=" in the '${PRIOR_BRANCH}' branch" if [[ ${DEBUG} -gt 0 ]]; then @@ -947,7 +1083,6 @@ handle_prior_website() fi if [[ -n ${NEWEST_VERSION} ]]; then - display_msg "${LOG_TYPE}" debug "Comparing prior Website ${PV} to newest ${NEWEST_VERSION}${B}" if [[ -z ${PRIOR_VERSION} || ${PRIOR_VERSION} < "${NEWEST_VERSION}" ]]; then MSG="There is a newer Allsky Website available${B}; please upgrade to it." MSG="${MSG}\nYour version: ${PV}" @@ -957,6 +1092,8 @@ handle_prior_website() MSG="${MSG}\nafter this installation finishes." display_msg --log notice "${MSG}" echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" + else + display_msg "${LOG_TYPE}" info "Prior local Website already at ${NEWEST_VERSION}${B}" fi fi } @@ -964,9 +1101,9 @@ handle_prior_website() #### # Get the locale, prompting if we can't determine it. -LOCALE="" +DESIRED_LOCALE="" CURRENT_LOCALE="" -get_locale() +get_desired_locale() { # A lot of people have the incorrect locale so prompt for the correct one. @@ -985,21 +1122,24 @@ get_locale() MSG="${MSG}\n\nIt will take a moment for the locale(s) to be installed." MSG="${MSG}\n\nWhen that is completed, rerun the Allsky installation." display_msg --log error "${MSG}" - exit_installation 1 + + exit_installation 1 "${STATUS_NO_LOCALE}" "None on system." fi [[ ${DEBUG} -gt 1 ]] && display_msg --logonly debug "INSTALLED_LOCALES=${INSTALLED_LOCALES}" - # If the prior version of Allsky had a locale set but it's not - # an installed one, let the user know. + # If the prior version of Allsky had a locale set but it's no longer installed, + # let the user know. # This can happen if they use the settings file from a different Pi or different OS. local MSG2="" - if [[ -n ${PRIOR_ALLSKY} && -n ${PRIOR_SETTINGS_FILE} ]]; then - local L="$( settings .locale "${PRIOR_SETTINGS_FILE}" )" - if [[ ${L} != "" && ${L} != "null" ]]; then - local X="$(echo "${INSTALLED_LOCALES}" | grep "${L}")" + if [[ -z ${DESIRED_LOCALE} && -n ${PRIOR_ALLSKY} && -n ${PRIOR_SETTINGS_FILE} ]]; then + # People rarely change locale once set, so assume they still want the prior one. + DESIRED_LOCALE="$( settings .locale "${PRIOR_SETTINGS_FILE}" )" + if [[ -n ${DESIRED_LOCALE} && ${DESIRED_LOCALE} != "null" ]]; then + local X="$(echo "${INSTALLED_LOCALES}" | grep "${DESIRED_LOCALE}")" if [[ -z ${X} ]]; then - MSG2="NOTE: Your prior locale (${L}) is not installed on this Pi." + # This is probably EXTREMELY rare. + MSG2="NOTE: Your prior locale (${DESIRED_LOCALE}) is no longer installed on this Pi." fi fi fi @@ -1014,7 +1154,7 @@ get_locale() CURRENT_LOCALE="$(echo "${TEMP_LOCALE}" | sed --silent -e '/LC_ALL=/ s/LC_ALL=//p')" fi fi - display_msg --logonly debug "TEMP_LOCALE=${TEMP_LOCALE}, CURRENT_LOCALE=${CURRENT_LOCALE}" + display_msg --logonly info "CURRENT_LOCALE=${CURRENT_LOCALE}, TEMP_LOCALE=[[ ${TEMP_LOCALE} ]]" local D="" if [[ -n ${CURRENT_LOCALE} && ${CURRENT_LOCALE} != "null" ]]; then @@ -1022,11 +1162,19 @@ get_locale() else CURRENT_LOCALE="" fi + STATUS_VARIABLES+=("CURRENT_LOCALE='${CURRENT_LOCALE}'\n") + + # If they had a locale from the prior Allsky and it's still here, use it; no need to prompt. + if [[ -n ${DESIRED_LOCALE} && ${DESIRED_LOCALE} == "${CURRENT_LOCALE}" ]]; then + STATUS_VARIABLES+=("get_desired_locale='true'\n") + STATUS_VARIABLES+=("DESIRED_LOCALE='${DESIRED_LOCALE}'\n") + return + fi MSG="\nSelect your locale; the default is highlighted in red." - MSG="${MSG}\nIf it's not in the list, press ." + MSG="${MSG}\nIf your desired locale is not in the list, press ." MSG="${MSG}\n\nIf you change the locale, the system will reboot and" - MSG="${MSG}\nyou will need to restart the installation." + MSG="${MSG}\nyou will need to continue the installation." [[ -n ${MSG2} ]] && MSG="${MSG}\n\n${MSG2}" # This puts in IL the necessary strings to have whiptail display what looks like @@ -1039,69 +1187,83 @@ get_locale() done #shellcheck disable=SC2086 - LOCALE=$(whiptail --title "${TITLE}" ${D} --menu "${MSG}" 25 "${WT_WIDTH}" 4 "${IL[@]}" \ + DESIRED_LOCALE=$(whiptail --title "${TITLE}" ${D} --menu "${MSG}" 25 "${WT_WIDTH}" 4 "${IL[@]}" \ 3>&1 1>&2 2>&3) - if [[ -z ${LOCALE} ]]; then + if [[ -z ${DESIRED_LOCALE} ]]; then MSG="You need to set the locale before the installation can run." - MSG="${MSG}\nIf your locale was not in the list, run 'raspi-config' to update the list," - MSG="${MSG}\nthen rerun the installation." + MSG="${MSG}\n If your desired locale was not in the list," + MSG="${MSG}\n run 'raspi-config' to update the list, then rerun the installation." display_msg info "${MSG}" display_msg --logonly info "No locale selected; exiting." - exit_installation 0 - elif echo "${LOCALE}" | grep --silent "Box options" ; then - # Got a usage message from whiptail. + + exit_installation 0 "${STATUS_NOT_CONTINUE}" "Locale(s) available but none selected." + + elif echo "${DESIRED_LOCALE}" | grep --silent "Box options" ; then + # Got a usage message from whiptail. This happened once so I added this check. # Must be no space between the last double quote and ${INSTALLED_LOCALES}. #shellcheck disable=SC2086 MSG="Got usage message from whiptail: D='${D}', INSTALLED_LOCALES="${INSTALLED_LOCALES} - MSG="${MSG}\nFix the problem and try the installation again." + MSG="${MSG}\n Fix the problem and try the installation again." display_msg --log error "${MSG}" - exit_installation 1 + exit_installation 1 "${STATUS_ERROR}" "Got usage message from whitail." fi + + STATUS_VARIABLES+=("get_desired_locale='true'\n") + STATUS_VARIABLES+=("DESIRED_LOCALE='${DESIRED_LOCALE}'\n") } -update_locale() -{ - local L="${1}" # locale - local S="${2}" # settings file - jq ".locale = \"${L}\"" "${S}" > /tmp/x && mv /tmp/x "${S}" -} #### # Set the locale set_locale() { - # ${LOCALE} and ${CURRENT_LOCALE} are set + # ${DESIRED_LOCALE} and ${CURRENT_LOCALE} are already set - if [[ ${CURRENT_LOCALE} == "${LOCALE}" ]]; then - display_msg --log progress "Keeping '${LOCALE}' locale." + if [[ ${CURRENT_LOCALE} == "${DESIRED_LOCALE}" ]]; then + display_msg --log progress "Keeping '${DESIRED_LOCALE}' locale." local L="$( settings .locale )" + MSG="Settings file '${SETTINGS_FILE}'" if [[ ${L} == "" || ${L} == "null" ]]; then - # Probably a new install. - MSG="Info: Settings file '${SETTINGS_FILE}' did not contain .locale." + # Either a new install or an upgrade from an older Allsky. + MSG="${MSG} did NOT contain .locale so adding it." display_msg --logonly info "${MSG}" - update_locale "${LOCALE}" "${SETTINGS_FILE}" + update_json_file ".locale" "${DESIRED_LOCALE}" "${SETTINGS_FILE}" + +# TODO: Something appears to still be unlinking the settings file +# from its camera-specific file, so do "ls" of the settings +# files to try and pinpoint the problem. +#shellcheck disable=SC2012 +MSG="$( /bin/ls -l "${ALLSKY_CONFIG}/settings"*.json 2>/dev/null | sed 's/^/ /' )" +display_msg --logonly info "Settings files now:\n${MSG}" + else - MSG="Info: Settings file '${SETTINGS_FILE}' contained .locale = '${L}'." + MSG="${MSG} CONTAINED .locale = '${L}'." display_msg --logonly info "${MSG}" fi + STATUS_VARIABLES+=("set_locale='true'\n") return fi - display_msg --log progress "Setting locale to '${LOCALE}'." - update_locale "${LOCALE}" "${SETTINGS_FILE}" + display_msg --log progress "Setting locale to '${DESIRED_LOCALE}'." + update_json_file ".locale" "${DESIRED_LOCALE}" "${SETTINGS_FILE}" + +# TODO: same as above... +#shellcheck disable=SC2012 +MSG="$( /bin/ls -l "${ALLSKY_CONFIG}/settings"*.json 2>/dev/null | sed 's/^/ /' )" +display_msg --logonly info "Settings files now:\n${MSG}" # This updates /etc/default/locale - sudo update-locale LC_ALL="${LOCALE}" LANGUAGE="${LOCALE}" LANG="${LOCALE}" + sudo update-locale LC_ALL="${DESIRED_LOCALE}" LANGUAGE="${DESIRED_LOCALE}" LANG="${DESIRED_LOCALE}" if ask_reboot "locale" ; then - display_msg --logonly info "Rebooting to set locale to '${LOCALE}'" - do_reboot # does not return + display_msg --logonly info "Rebooting to set locale to '${DESIRED_LOCALE}'" + do_reboot "${STATUS_LOCALE_REBOOT}" "" # does not return fi display_msg warning "You must reboot before continuing with the installation." display_msg --logonly info "User elected not to reboot to update locale." - exit_installation 0 + exit_installation 0 "${STATUS_NO_REBOOT}" "to update locale." } @@ -1109,47 +1271,56 @@ set_locale() # See if a prior Allsky exists; if so, set some variables. does_prior_Allsky_exist() { + if [[ ! -d ${PRIOR_ALLSKY_DIR}/src ]]; then + display_msg --logonly info "No prior Allsky found." + return 1 + fi + PRIOR_ALLSKY="" PRIOR_CAMERA_TYPE="" - if [[ -d ${PRIOR_ALLSKY_DIR}/src ]]; then - PRIOR_ALLSKY_VERSION="$( get_version "${PRIOR_ALLSKY_DIR}/" )" - if [[ -n ${PRIOR_ALLSKY_VERSION} ]]; then - case "${PRIOR_ALLSKY_VERSION}" in - "v2022.03.01") # First Allsky version with a version file - # This is an old style Allsky with ${CAMERA} in config.sh and - # the first version with a "version" file. - # Don't do anything here; go to the "if" after the "esac". - ;; - - *) - # Newer version. - # PRIOR_SETTINGS_FILE should be a link to a camera-specific settings file. - PRIOR_ALLSKY="new" - PRIOR_SETTINGS_FILE="${PRIOR_CONFIG_DIR}/${SETTINGS_FILE_NAME}" - PRIOR_CAMERA_TYPE="$(get_variable "CAMERA_TYPE" "${PRIOR_CONFIG_FILE}")" + PRIOR_CAMERA_MODEL="" + PRIOR_ALLSKY_VERSION="$( get_version "${PRIOR_ALLSKY_DIR}/" )" + if [[ -n ${PRIOR_ALLSKY_VERSION} ]]; then + display_msg --logonly info "Prior Allsky version ${PRIOR_ALLSKY_VERSION} found." + case "${PRIOR_ALLSKY_VERSION}" in + "v2022.03.01") # First Allsky version with a "version" file + # This is an old style Allsky with ${CAMERA} in config.sh. + # Don't do anything here; go to the "if" after the "esac". + ;; - # This shouldn't happen, but just in case ... - [[ ! -f ${PRIOR_SETTINGS_FILE} ]] && PRIOR_SETTINGS_FILE="" - ;; - esac - fi + *) + # Newer version. + # PRIOR_SETTINGS_FILE should be a link to a camera-specific settings file. + PRIOR_ALLSKY="newStyle" + PRIOR_SETTINGS_FILE="${PRIOR_CONFIG_DIR}/${SETTINGS_FILE_NAME}" + if [[ -f ${PRIOR_SETTINGS_FILE} ]]; then + PRIOR_CAMERA_TYPE="$( settings ".cameraType" "${PRIOR_SETTINGS_FILE}" )" + PRIOR_CAMERA_MODEL="$( settings ".cameraModel" "${PRIOR_SETTINGS_FILE}" )" + else + # This shouldn't happen... + PRIOR_SETTINGS_FILE="" + display_msg --log warning "No prior new style settings file found!" + fi - if [[ -z ${PRIOR_ALLSKY} ]]; then - PRIOR_ALLSKY="old" - PRIOR_ALLSKY_VERSION="${PRIOR_ALLSKY_VERSION:-old}" - local CAMERA="$(get_variable "CAMERA" "${PRIOR_CONFIG_FILE}")" - PRIOR_CAMERA_TYPE="$( CAMERA_to_CAMERA_TYPE "${CAMERA}" )" - PRIOR_SETTINGS_FILE="${OLD_RASPAP_DIR}/settings_${CAMERA}.json" - [[ ! -f ${PRIOR_SETTINGS_FILE} ]] && PRIOR_SETTINGS_FILE="" - fi + ;; + esac + fi - display_msg "${LOG_TYPE}" info "PRIOR_ALLSKY_VERSION=${PRIOR_ALLSKY_VERSION}" - display_msg "${LOG_TYPE}" info "PRIOR_CAMERA_TYPE=${PRIOR_CAMERA_TYPE}" - display_msg "${LOG_TYPE}" info "PRIOR_SETTINGS_FILE=${PRIOR_SETTINGS_FILE}" - return 0 - else - return 1 + if [[ -z ${PRIOR_ALLSKY} ]]; then + PRIOR_ALLSKY="oldStyle" + PRIOR_ALLSKY_VERSION="${PRIOR_ALLSKY_VERSION:-old}" + local CAMERA="$( get_variable "CAMERA" "${PRIOR_CONFIG_FILE}" )" + PRIOR_CAMERA_TYPE="$( CAMERA_to_CAMERA_TYPE "${CAMERA}" )" + # PRIOR_CAMERA_MODEL wasn't stored anywhere so can't set it. + PRIOR_SETTINGS_FILE="${OLD_RASPAP_DIR}/settings_${CAMERA}.json" + [[ ! -f ${PRIOR_SETTINGS_FILE} ]] && PRIOR_SETTINGS_FILE="" fi + + display_msg "${LOG_TYPE}" info "PRIOR_ALLSKY_VERSION=${PRIOR_ALLSKY_VERSION}" + display_msg "${LOG_TYPE}" info "PRIOR_CAMERA_TYPE=${PRIOR_CAMERA_TYPE}" + display_msg "${LOG_TYPE}" info "PRIOR_SETTINGS_FILE=${PRIOR_SETTINGS_FILE}" + + return 0 } @@ -1159,12 +1330,16 @@ does_prior_Allsky_exist() # Look for a directory inside the old one to make sure it's really an old allsky. prompt_for_prior_Allsky() { + if [[ -n ${PRIOR_ALLSKY} ]]; then + STATUS_VARIABLES+=("prompt_for_prior_Allsky='true'\n") MSG="You have a prior version of Allsky in ${PRIOR_ALLSKY_DIR}." MSG="${MSG}\n\nDo you want to restore the prior images, darks, and certain settings?" if whiptail --title "${TITLE}" --yesno "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then # Set the prior camera type to the new, default camera type. CAMERA_TYPE="${PRIOR_CAMERA_TYPE}" + STATUS_VARIABLES+=("CAMERA_TYPE='${CAMERA_TYPE}'\n") + display_msg --log info "Will restore from prior version of Allsky." return 0 else CAMERA_TYPE="" @@ -1181,15 +1356,17 @@ prompt_for_prior_Allsky() MSG="${MSG}\n\nIf you DO have a prior version and you want images, darks, and certain settings moved from the prior version to the new one, rename the prior version to ${PRIOR_ALLSKY_DIR} before running this installation." MSG="${MSG}\n\nDo you want to continue?" if ! whiptail --title "${TITLE}" --yesno "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then - MSG="Rename the directory with your prior version of Allsky to\n" - MSG="${MSG}\n '${PRIOR_ALLSKY_DIR}', then run the installation again.\n" + MSG="Rename the directory with your prior version of Allsky to" + MSG="${MSG}\n '${PRIOR_ALLSKY_DIR}', then run the installation again." display_msg --log info "${MSG}" - exit_installation 0 + exit_installation 0 "${STATUS_NOT_CONTINUE}" "after no prior Allsky was found." fi + STATUS_VARIABLES+=("prompt_for_prior_Allsky='true'\n") fi # No prior Allsky so force creating a default settings file. - FORCE_CREATING_SETTINGS_FILE="true" + FORCE_CREATING_DEFAULT_SETTINGS_FILE="true" + STATUS_VARIABLES+=("FORCE_CREATING_DEFAULT_SETTINGS_FILE='${FORCE_CREATING_DEFAULT_SETTINGS_FILE}'\n") } @@ -1203,19 +1380,23 @@ install_dependencies_etc() TMP="${ALLSKY_INSTALLATION_LOGS}/make_deps.log" #shellcheck disable=SC2024 sudo make deps > "${TMP}" 2>&1 - check_success $? "Dependency installation failed" "${TMP}" "${DEBUG}" || exit_with_image 1 + check_success $? "Dependency installation failed" "${TMP}" "${DEBUG}" + [[ $? -ne 0 ]] && exit_with_image 1 "${STATUS_ERROR}" "dependency installation failed" display_msg --log progress "Preparing Allsky commands." TMP="${ALLSKY_INSTALLATION_LOGS}/make_all.log" #shellcheck disable=SC2024 make all > "${TMP}" 2>&1 - check_success $? "Compile failed" "${TMP}" "${DEBUG}" || exit_with_image 1 + check_success $? "Compile failed" "${TMP}" "${DEBUG}" + [[ $? -ne 0 ]] && exit_with_image 1 "${STATUS_ERROR}" "compile failed" TMP="${ALLSKY_INSTALLATION_LOGS}/make_install.log" #shellcheck disable=SC2024 sudo make install > "${TMP}" 2>&1 - check_success $? "make install failed" "${TMP}" "${DEBUG}" || exit_with_image 1 + check_success $? "make install failed" "${TMP}" "${DEBUG}" + [[ $? -ne 0 ]] && exit_with_image 1 "${STATUS_ERROR}" "make insall_failed" + STATUS_VARIABLES+=("install_dependencies_etc='true'\n") return 0 } @@ -1225,7 +1406,7 @@ install_dependencies_etc() update_config_sh() { local C="${ALLSKY_CONFIG}/config.sh" - display_msg --log progress "Updating '${C}'" + display_msg --log progress "Updating some '${C}' variables." if [[ -z ${ALLSKY_VERSION} ]]; then display_msg --log error "ALLSKY_VERSION is empty in update_config_sh()" fi @@ -1237,6 +1418,8 @@ update_config_sh() -e "s;^ALLSKY_VERSION=.*$;ALLSKY_VERSION=\"${ALLSKY_VERSION}\";" \ -e "s;^CAMERA_TYPE=.*$;CAMERA_TYPE=\"${CAMERA_TYPE}\";" \ "${C}" + + STATUS_VARIABLES+=( "update_config_sh='true'\n" ) } @@ -1272,7 +1455,7 @@ prompt_for_lat_long() break else if VALUE="$( convertLatLong "${VALUE}" "${TYPE}" 2>&1 )" ; then - jq ".${TYPE}=\"${VALUE}\" " "${SETTINGS_FILE}" > /tmp/x && mv /tmp/x "${SETTINGS_FILE}" + update_json_file ".${TYPE}" "${VALUE}" "${SETTINGS_FILE}" display_msg --log progress "${HUMAN_TYPE} set to ${VALUE}." echo "${VALUE}" break @@ -1314,192 +1497,227 @@ get_lat_long() #### # Convert the prior settings file to a new one, # then link the new one to the camera-specific name. -convert_settings() # prior_version, prior_file, current_version +convert_settings() # prior_version, new_version, prior_file, new_file { -cat > /dev/null <&1 1>&2 2>&3 + display_msg info "\n${MSG}\n" + echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" + display_msg --logonly info "Settings from ${PRIOR_ALLSKY_VERSION} copied over." ;; *) # This could be one of many old versions of Allsky, # so don't try to copy all the settings since there have # been many changes, additions, and deletions. - # As far as I know, latitude and longitude have never changed, + # As far as I know, latitude, longitude, and angle have never changed names, # and are required and have no default, # so try to restore them so Allsky can restart automatically. - LAT="$(settings .latitude "${PRIOR_SETTINGS_FILE}")" - LONG="$(settings .longitude "${PRIOR_SETTINGS_FILE}")" - ANGLE="$(settings .angle "${PRIOR_SETTINGS_FILE}")" - jq ".latitude=\"${LAT}\" | .longitude=\"${LONG}\" | .angle=\"${ANGLE}\"" \ - "${SETTINGS_FILE}" > /tmp/x && mv /tmp/x "${SETTINGS_FILE}" + local LAT="$(settings .latitude "${PRIOR_SETTINGS_FILE}")" + update_json_file ".latitude" "${LAT}" "${SETTINGS_FILE}" + local LONG="$(settings .longitude "${PRIOR_SETTINGS_FILE}")" + update_json_file ".longitude" "${LONG}" "${SETTINGS_FILE}" + local ANGLE="$(settings .angle "${PRIOR_SETTINGS_FILE}")" + update_json_file ".angle" "${ANGLE}" "${SETTINGS_FILE}" display_msg --log progress "Prior latitude, longitude, and angle restored." MSG="You need to manually transfer your old settings to the WebUI.\n" @@ -1507,26 +1725,32 @@ restore_prior_settings_files() MSG="${MSG} since you last installed Allsky, so it will likely be easiest" MSG="${MSG} to re-enter everything via the WebUI's 'Allsky Settings' page." whiptail --title "${TITLE}" --msgbox "${MSG}" 18 "${WT_WIDTH}" 3>&1 1>&2 2>&3 - display_msg --log info "\n${MSG}\n" + display_msg info "\n${MSG}\n" echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" + display_msg --logonly info "Only a few settings from very old ${PRIOR_ALLSKY_VERSION} copied over." ;; - esac - else - # This should "never" happen. - # They have a prior Allsky version but no "settings file? - display_msg --log error "Prior settings file missing: ${PRIOR_SETTINGS_FILE}." + # Set to null to force the user to look at the settings before Allsky will run. + update_json_file ".lastChanged" "" "${SETTINGS_FILE}" - # If we ever automate migrating settings, this next statement should be deleted. - FORCE_CREATING_SETTINGS_FILE="true" + RESTORED_PRIOR_SETTINGS_FILE="true" + FORCE_CREATING_DEFAULT_SETTINGS_FILE="false" + else + # First time through there often won't be SETTINGS_FILE. + display_msg --logonly info "No new settings file yet..." + FORCE_CREATING_DEFAULT_SETTINGS_FILE="true" fi fi + + STATUS_VARIABLES+=( "RESTORED_PRIOR_SETTINGS_FILE='${RESTORED_PRIOR_SETTINGS_FILE}'\n" ) } #### # If the user wanted to restore files from a prior version of Allsky, do that. restore_prior_files() { + STATUS_VARIABLES+=( "restore_prior_files='true'\n" ) + if [[ -d ${OLD_RASPAP_DIR} ]]; then MSG="\nThe '${OLD_RASPAP_DIR}' directory is no longer used.\n" MSG="${MSG}When installation is done you may remove it by executing:\n" @@ -1542,9 +1766,17 @@ restore_prior_files() return # Nothing left to do in this function, so return fi - # TODO: this script is going away in the next release. + # Do all the being restores, then all the updates. + local V="" + + display_msg --log progress "Restoring prior:" + + local SPACE=" " + local NOT_RESTORED="prior version does not exist so not restored" + # TODO: endOfNight_additionalStepts.sh script is going away in the next major release. + local ITEM="${SPACE}endOfNight_additionalSteps.sh" if [[ -f ${PRIOR_ALLSKY_DIR}/scripts/endOfNight_additionalSteps.sh ]]; then - display_msg --log progress "Restoring endOfNight_additionalSteps.sh." + display_msg --log progress "${ITEM}" cp -a "${PRIOR_ALLSKY_DIR}/scripts/endOfNight_additionalSteps.sh" "${ALLSKY_SCRIPTS}" MSG="The ${ALLSKY_SCRIPTS}/endOfNight_additionalSteps.sh file will be removed" @@ -1552,92 +1784,100 @@ restore_prior_files() MSG="${MSG}\nso please move your code to the 'Script' module in" MSG="${MSG}\nthe 'Night to Day Transition Flow' of the Module Manager." MSG="${MSG}\nSee the 'Explanations --> Module' documentation for more details." - display_msg --log info "\n${MSG}\n" + display_msg --log warning "\n${MSG}\n" echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" else - MSG="No prior 'endOfNight_additionalSteps.sh' so can't restore." - display_msg "${LOG_TYPE}" progress "${MSG}" + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi + ITEM="${SPACE}'images' directory" if [[ -d ${PRIOR_ALLSKY_DIR}/images ]]; then - display_msg --log progress "Restoring images." + display_msg --log progress "${ITEM}" mv "${PRIOR_ALLSKY_DIR}/images" "${ALLSKY_HOME}" else # This is probably very rare so let the user know - MSG="No prior 'images' directory so can't restore; This unusual." - display_msg --log info "${MSG}" + display_msg --log progress "${ITEM}: ${NOT_RESTORED}. This is unusual." fi + ITEM="${SPACE}'darks' directory" if [[ -d ${PRIOR_ALLSKY_DIR}/darks ]]; then - display_msg --log progress "Restoring darks." + display_msg --log progress "${ITEM}" mv "${PRIOR_ALLSKY_DIR}/darks" "${ALLSKY_HOME}" else - display_msg "${LOG_TYPE}" progress "No prior 'darks' directory so can't restore." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi + ITEM="${SPACE}'modules' directory" if [[ -d ${PRIOR_CONFIG_DIR}/modules ]]; then - display_msg --log progress "Restoring modules." + display_msg --log progress "${ITEM}" + + # Copy the user's prior data to the new file which may contain new fields. "${ALLSKY_SCRIPTS}"/flowupgrade.py --prior "${PRIOR_CONFIG_DIR}" --config "${ALLSKY_CONFIG}" else - display_msg "${LOG_TYPE}" progress "No prior 'modules' directory so can't restore." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi + ITEM="${SPACE}'overlay' directory" if [[ -d ${PRIOR_CONFIG_DIR}/overlay ]]; then - display_msg --log progress "Restoring overlays." + display_msg --log progress "${ITEM}" cp -ar "${PRIOR_CONFIG_DIR}/overlay" "${ALLSKY_CONFIG}" # Restore the fields.json file as it's part of the main Allsky distribution # and should be replaced during an upgrade. cp -ar "${ALLSKY_REPO}/overlay/config/fields.json" "${ALLSKY_OVERLAY}/config/" else - display_msg "${LOG_TYPE}" progress "No prior 'overlay' directory so can't restore." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" + fi + + ITEM="${SPACE}'ssl' directory" + if [[ -d ${PRIOR_CONFIG_DIR}/ssl ]]; then + display_msg --log progress "${ITEM}" + cp -ar "${PRIOR_CONFIG_DIR}/ssl" "${ALLSKY_CONFIG}" + else + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi # This is not in a "standard" directory so we need to determine where it was. - EXTRA="${PRIOR_ALLSKY_DIR}${ALLSKY_EXTRA//${ALLSKY_HOME}/}" + local EXTRA="${PRIOR_ALLSKY_DIR}${ALLSKY_EXTRA//${ALLSKY_HOME}/}" + ITEM="${SPACE}'${EXTRA}' directory" if [[ -d ${EXTRA} ]]; then - display_msg --log progress "Restoring 'extra' files." + display_msg --log progress "${ITEM}" cp -ar "${EXTRA}" "${ALLSKY_EXTRA}/.." else - display_msg "${LOG_TYPE}" progress "No prior 'extra' directory so can't restore." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi - if [[ ${PRIOR_ALLSKY} == "new" ]]; then + local D + if [[ ${PRIOR_ALLSKY} == "newStyle" ]]; then D="${PRIOR_CONFIG_DIR}" else # raspap.auth was in a different directory in older versions. D="${OLD_RASPAP_DIR}" fi + ITEM="${SPACE}WebUI security settings" if [[ -f ${D}/raspap.auth ]]; then - display_msg --log progress "Restoring WebUI security settings." + display_msg --log progress "${ITEM}" cp -a "${D}/raspap.auth" "${ALLSKY_CONFIG}" else - display_msg "${LOG_TYPE}" progress "No prior 'WebUI security settings' so can't restore." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi # Restore any REMOTE Allsky Website configuration file. + ITEM="${SPACE}'${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_NAME}'" if [[ -f ${PRIOR_CONFIG_DIR}/${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_NAME} ]]; then - MSG="Restoring remote Allsky Website ${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_NAME}." - display_msg --log progress "${MSG}" + display_msg --log progress "${ITEM}" cp -a "${PRIOR_CONFIG_DIR}/${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_NAME}" \ "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}" - # Update "AllskyVersion" if needed. + # Used below to update "AllskyVersion" if needed. V="$( settings .config.AllskyVersion "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}" )" - display_msg "${LOG_TYPE}" info "Prior remote Website's AllskyVersion=${V}" - if [[ ${V} != "${ALLSKY_VERSION}" ]]; then - MSG="Updating AllskyVersion in remote Website from '${V}' to '${ALLSKY_VERSION}'" - display_msg --log progress "${MSG}" - jq ".config.AllskyVersion = \"${ALLSKY_VERSION}\"" \ - "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}" > /tmp/x \ - && mv /tmp/x "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}" - fi +# TODO: is ConfigVersion still needed after the Website is part of Allsky? # Check if this is an older Allsky Website configuration file type. # The remote config file should have .ConfigVersion. local OLD="false" - NEW_CONFIG_VERSION="$(settings .ConfigVersion "${REPO_WEBCONFIG_FILE}")" - PRIOR_CONFIG_VERSION="$(settings .ConfigVersion "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}")" + local NEW_CONFIG_VERSION="$(settings .ConfigVersion "${REPO_WEBCONFIG_FILE}")" + local PRIOR_CONFIG_VERSION="$(settings .ConfigVersion "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}")" if [[ ${PRIOR_CONFIG_VERSION} == "" || ${PRIOR_CONFIG_VERSION} == "null" ]]; then OLD="true" # Hmmm, it should have the version MSG="Prior Website configuration file '${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}'" @@ -1656,60 +1896,57 @@ restore_prior_files() MSG="${MSG} to see what fields have been added, changed, or removed.\n" display_msg --log warning "${MSG}" echo -e "\n\n==========\n${MSG}" >> "${POST_INSTALLATION_ACTIONS}" + else + MSG="Remote Website .ConfigVersion is current @ ${NEW_CONFIG_VERSION}" + display_msg --logonly info "${MSG}" fi else # We don't check for old LOCAL Allsky Website configuration files. # That's done when they install the Allsky Website. - display_msg "${LOG_TYPE}" info "No prior remote Website configuration so can't restore." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" fi + ITEM="${SPACE}uservariables.sh" if [[ -f ${PRIOR_CONFIG_DIR}/uservariables.sh ]]; then - display_msg --log progress "Restoring uservariables.sh." + display_msg --log progress "${ITEM}: ${NOT_RESTORED}" cp -a "${PRIOR_CONFIG_DIR}/uservariables.sh" "${ALLSKY_CONFIG}" + # Don't bother with the "else" part since this file is very rarely used. fi - restore_prior_settings_files + restore_prior_settings_file # Do NOT restore options.json - it will be recreated. # See if the prior config.sh and ftp-setting.sh are the same version as # the new ones; if so, we can copy them to the new version. # Currently what's in ${ALLSKY_CONFIG} are copies of the repo files. + RESTORED_PRIOR_CONFIG_SH="false" # Global variable + RESTORED_PRIOR_FTP_SH="false" # Global variable - CONFIG_SH_VERSION="$(get_variable "CONFIG_SH_VERSION" "${ALLSKY_CONFIG}/config.sh")" - PRIOR_CONFIG_SH_VERSION="$(get_variable "CONFIG_SH_VERSION" "${PRIOR_CONFIG_FILE}")" - display_msg "${LOG_TYPE}" debug "CONFIG_SH_VERSION=${CONFIG_SH_VERSION}, PRIOR=${PRIOR_CONFIG_SH_VERSION}" + local CONFIG_SH_VERSION="$( get_variable "CONFIG_SH_VERSION" "${ALLSKY_CONFIG}/config.sh" )" + local PRIOR_CONFIG_SH_VERSION="$( get_variable "CONFIG_SH_VERSION" "${PRIOR_CONFIG_FILE}" )" + ITEM="${SPACE}'config.sh' file" if [[ ${CONFIG_SH_VERSION} == "${PRIOR_CONFIG_SH_VERSION}" ]]; then - RESTORED_PRIOR_CONFIG_SH="true" - display_msg --log progress "Restoring prior 'config.sh' file." - cp "${PRIOR_CONFIG_FILE}" "${ALLSKY_CONFIG}" - - local PRIOR="$( get_variable "ALLSKY_VERSION" "${PRIOR_CONFIG_FILE}" )" - if [[ ${PRIOR} != "${ALLSKY_VERSION}" ]]; then - MSG="Updating ALLSKY_VERSION in 'config.sh' to '${ALLSKY_VERSION}'." - sed -i "/ALLSKY_VERSION=/ c ALLSKY_VERSION=\"${ALLSKY_VERSION}\"" "${PRIOR_CONFIG_FILE}" - display_msg --log progress "${MSG}" - else - MSG="ALLSKY_VERSION (${PRIOR}) in prior config.sh same as new version." - display_msg --logonly info "${MSG}" - fi + display_msg --log progress "${ITEM}, as is" + cp "${PRIOR_CONFIG_FILE}" "${ALLSKY_CONFIG}" && RESTORED_PRIOR_CONFIG_SH="true" else - RESTORED_PRIOR_CONFIG_SH="false" if [[ -z ${PRIOR_CONFIG_SH_VERSION} ]]; then MSG="no prior version specified" else - # This is hopefully the last version with config.sh so don't + # v2023.05.01 is hopefully the last version with config.sh so don't # bother writing a function to convert from the prior version to this. MSG="prior version is old (${PRIOR_CONFIG_SH_VERSION})" fi - MSG="Not restoring 'config.sh': ${MSG}." - display_msg --log info "${MSG}" + display_msg --log progress "${ITEM}: ${NOT_RESTORED}: ${MSG}" fi + MSG=" CONFIG_SH_VERSION=${CONFIG_SH_VERSION}, PRIOR=${PRIOR_CONFIG_SH_VERSION}" + display_msg "${LOG_TYPE}" info "${MSG}" # Unlike the config.sh file which was always in allsky/config, # the ftp-settings.sh file used to be in allsky/scripts. # Get the current and prior (if any) file version. - FTP_SH_VERSION="$( get_variable "FTP_SH_VERSION" "${ALLSKY_CONFIG}/ftp-settings.sh" )" + local FTP_SH_VERSION="$( get_variable "FTP_SH_VERSION" "${ALLSKY_CONFIG}/ftp-settings.sh" )" + local PRIOR_FTP_SH_VERSION if [[ -f ${PRIOR_FTP_FILE} ]]; then # Allsky v2022.03.01 and newer. v2022.03.01 doesn't have FTP_SH_VERSION. PRIOR_FTP_SH_VERSION="$( get_variable "FTP_SH_VERSION" "${PRIOR_FTP_FILE}" )" @@ -1723,31 +1960,58 @@ restore_prior_files() PRIOR_FTP_FILE="" PRIOR_FTP_SH_VERSION="no file" fi - display_msg "${LOG_TYPE}" debug "FTP_SH_VERSION=${FTP_SH_VERSION}, PRIOR=${PRIOR_FTP_SH_VERSION}" + ITEM="${SPACE}ftp-settings.sh" if [[ ${FTP_SH_VERSION} == "${PRIOR_FTP_SH_VERSION}" ]]; then - RESTORED_PRIOR_FTP_SH="true" - display_msg --log progress "Restoring prior 'ftp-settings.sh' file." - cp "${PRIOR_FTP_FILE}" "${ALLSKY_CONFIG}" + display_msg --log progress "${ITEM}, as is" + cp "${PRIOR_FTP_FILE}" "${ALLSKY_CONFIG}" && RESTORED_PRIOR_FTP_SH="true" else - RESTORED_PRIOR_FTP_SH="false" if [[ ${PRIOR_FTP_SH_VERSION} == "no version" ]]; then - MSG="unknown prior FTP_SH_VERSION." + MSG=": unknown prior FTP_SH_VERSION" elif [[ ${PRIOR_FTP_SH_VERSION} == "old" ]]; then - MSG="old location so no FTP_SH_VERSION." - elif [[ ${PRIOR_FTP_SH_VERSION} == "no file" ]]; then - MSG="no prior file." + MSG=": old location so no FTP_SH_VERSION" + elif [[ ${PRIOR_FTP_SH_VERSION} != "no file" ]]; then + MSG=": unknown PRIOR_FTP_SH_VERSION: '${PRIOR_FTP_SH_VERSION}'" + fi + display_msg --log progress "${ITEM}: ${NOT_RESTORED}${MSG}" + fi + MSG=" FTP_SH_VERSION=${FTP_SH_VERSION}, PRIOR=${PRIOR_FTP_SH_VERSION}" + display_msg "${LOG_TYPE}" info "${MSG}" + + # Done with restores, now the updates. + + if [[ -f ${PRIOR_CONFIG_DIR}/${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_NAME} ]]; then + if [[ ${V} != "${ALLSKY_VERSION}" ]]; then + MSG="Updating AllskyVersion in remote Website from '${V}' to '${ALLSKY_VERSION}'" + display_msg --log progress "${MSG}" + update_json_file ".config.AllskyVersion" "${ALLSKY_VERSION}" \ + "${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE}" + else + display_msg --log progress "Prior remote Website already at latest Allsky version ${V}." + fi + fi + + if [[ ${CONFIG_SH_VERSION} == "${PRIOR_CONFIG_SH_VERSION}" ]]; then + # This version should be the same as the what's in the prior "version" file. + local PRIOR="$( get_variable "ALLSKY_VERSION" "${PRIOR_CONFIG_FILE}" )" + if [[ ${PRIOR} != "${ALLSKY_VERSION}" ]]; then + MSG="Updating ALLSKY_VERSION in 'config.sh' to '${ALLSKY_VERSION}'." + sed -i "/ALLSKY_VERSION=/ c ALLSKY_VERSION=\"${ALLSKY_VERSION}\"" "${PRIOR_CONFIG_FILE}" + display_msg --log progress "${MSG}" else - MSG="unknown PRIOR_FTP_SH_VERSION: '${PRIOR_FTP_SH_VERSION}'." + MSG="ALLSKY_VERSION (${PRIOR}) in prior config.sh same as new version." + display_msg --logonly info "${MSG}" fi - display_msg --log progress "Not restoring prior 'ftp-settings.sh': ${MSG}" fi + STATUS_VARIABLES+=( "RESTORED_PRIOR_CONFIG_SH='${RESTORED_PRIOR_CONFIG_SH}'\n" ) + STATUS_VARIABLES+=( "RESTORED_PRIOR_FTP_SH='${RESTORED_PRIOR_FTP_SH}'\n" ) + if [[ ${RESTORED_PRIOR_CONFIG_SH} == "true" && ${RESTORED_PRIOR_FTP_SH} == "true" ]]; then return 0 fi - if [[ ${PRIOR_ALLSKY} == "new" ]]; then + if [[ ${PRIOR_ALLSKY} == "newStyle" ]]; then # The prior versions are similar to the new ones. MSG="" # If it has a version number it's probably close to the current version. @@ -1799,14 +2063,15 @@ restore_prior_files() # This can be needed if the user hosed something up, or there was a problem somewhere. do_update() { +# TODO: get from settings file #shellcheck disable=SC2086,SC1091 # file doesn't exist in GitHub source "${ALLSKY_CONFIG}/config.sh" || exit ${ALLSKY_ERROR_STOP} # Get current CAMERA_TYPE if [[ -z ${CAMERA_TYPE} ]]; then display_msg --log error "CAMERA_TYPE not set in config.sh." - exit_installation 1 + exit_installation 1 "${STATUS_ERROR}" "No CAMERA_TYPE in config.sh during update." fi - create_webui_defines + [[ ${create_webui_defines} != "true" ]] && create_webui_defines save_camera_capabilities "false" || exit 1 set_permissions @@ -1817,13 +2082,15 @@ do_update() if ! grep --silent "/date" "${FINAL_SUDOERS_FILE}" ; then display_msg --log progress "Updating sudoers list." if ! grep --silent "/date" "${REPO_SUDOERS_FILE}" ; then - display_msg --log error "Please get the newest '$(basename "${REPO_SUDOERS_FILE}")' file from Git and try again." - exit_installation 2 + local F="$( basename "${REPO_SUDOERS_FILE}" )" + MSG="Please get the newest '${F}' file from Git and try again." + display_msg --log error "${MSG}" + exit_installation 2 "${STATUS_ERROR}" "'${F}' file is old." fi do_sudoers fi - exit_installation 0 + exit_installation 0 "${STATUS_OK}" "Update completed." } @@ -1831,55 +2098,92 @@ do_update() # Install the overlay and modules system install_overlay() { + if [[ ${installing_PHP_modules} != "true" ]]; then + display_msg --log progress "Installing PHP modules." + TMP="${ALLSKY_INSTALLATION_LOGS}/PHP_modules.log" + sudo apt-get --assume-yes install php-zip php-sqlite3 python3-pip > "${TMP}" 2>&1 + check_success $? "PHP module installation failed" "${TMP}" "${DEBUG}" + [[ $? -ne 0 ]] && exit_with_image 1 "${STATUS_ERROR}" "PHP module install failed." - display_msg --log progress "Installing PHP Modules." - TMP="${ALLSKY_INSTALLATION_LOGS}/PHP_modules.log" - sudo apt-get --assume-yes install php-zip php-sqlite3 python3-pip > "${TMP}" 2>&1 - check_success $? "PHP module installation failed" "${TMP}" "${DEBUG}" || exit_with_image 1 - - display_msg --log progress "Installing other PHP dependencies." - TMP="${ALLSKY_INSTALLATION_LOGS}/libatlas.log" - sudo apt-get --assume-yes install libatlas-base-dev > "${TMP}" 2>&1 - check_success $? "PHP dependencies failed" "${TMP}" "${DEBUG}" || exit_with_image 1 + display_msg --log progress "Installing other PHP dependencies." + TMP="${ALLSKY_INSTALLATION_LOGS}/libatlas.log" + sudo apt-get --assume-yes install libatlas-base-dev > "${TMP}" 2>&1 + check_success $? "PHP dependencies failed" "${TMP}" "${DEBUG}" + [[ $? -ne 0 ]] && exit_with_image 1 "${STATUS_ERROR}" "PHP dependencies failed." + STATUS_VARIABLES+=( "installing_PHP_modules='true'\n" ) + fi # Doing all the python dependencies at once can run /tmp out of space, so do one at a time. # This also allows us to display progress messages. if [[ ${OS} == "buster" ]]; then M=" for Buster" R="-buster" + # + # Force pip upgrade, without this installations on Buster fail + # + pip3 install --upgrade pip > /dev/null 2>&1 else M="" R="" fi - MSG2="\n\tThis may take a LONG time if the packages are not already installed." - display_msg --log progress "Installing Python dependencies${M}." "${MSG2}" TMP="${ALLSKY_INSTALLATION_LOGS}/Python_dependencies" - PIP3_BUILD="${ALLSKY_HOME}/pip3.build" - mkdir -p "${PIP3_BUILD}" + display_msg --log progress "Installing Python dependencies${M}:" COUNT=0 local NUM=$(wc -l < "${ALLSKY_REPO}/requirements${R}.txt") + : > "${STATUS_FILE_TEMP}" while read -r package do ((COUNT++)) echo "${package}" > /tmp/package + if [[ ${COUNT} -lt 10 ]]; then + C=" ${COUNT}" + else + C="${COUNT}" + fi + + local PACKAGE=" === Package # ${C} of ${NUM}: [${package}]" + # Need indirection since the ${STATUS_NAME} is the variable name and we want its value. + local STATUS_NAME="Python_dependency_${COUNT}" + eval "STATUS_VALUE=\${${STATUS_NAME}}" + if [[ ${STATUS_VALUE} == "true" ]]; then + display_msg --log progress "${PACKAGE} - already installed." + continue + fi + display_msg --log progress "${PACKAGE}" + L="${TMP}.${COUNT}.log" - display_msg --log progress " === Package # ${COUNT} of ${NUM}: [${package}]" - pip3 install --no-warn-script-location --build "${PIP3_BUILD}" -r /tmp/package > "${L}" 2>&1 + local M="Python dependency [${package}] failed" + pip3 install --no-warn-script-location -r /tmp/package > "${L}" 2>&1 # These files are too big to display so pass in "0" instead of ${DEBUG}. - if ! check_success $? "Python dependency [${package}] failed" "${L}" 0 ; then + if ! check_success $? "${M}" "${L}" 0 ; then rm -fr "${PIP3_BUILD}" - exit_with_image 1 + + # Add current status + update_status_from_temp_file + + exit_with_image 1 "${STATUS_ERROR}" "${M}." fi + echo "${STATUS_NAME}='true'" >> "${STATUS_FILE_TEMP}" done < "${ALLSKY_REPO}/requirements${R}.txt" - rm -fr "${PIP3_BUILD}" - display_msg --log progress "Installing Trutype fonts." - TMP="${ALLSKY_INSTALLATION_LOGS}/msttcorefonts.log" - sudo apt-get --assume-yes install msttcorefonts > "${TMP}" 2>&1 - check_success $? "Trutype fonts failed" "${TMP}" "${DEBUG}" || exit_with_image 1 + # Add the status back in. + update_status_from_temp_file - display_msg --log progress "Setting up modules and overlays." + if [[ ${installing_Trutype_fonts} != "true" ]]; then + display_msg --log progress "Installing Trutype fonts." + TMP="${ALLSKY_INSTALLATION_LOGS}/msttcorefonts.log" + local M="Trutype fonts failed" + sudo apt-get --assume-yes install msttcorefonts > "${TMP}" 2>&1 + check_success $? "${M}" "${TMP}" "${DEBUG}" || exit_with_image 1 "${STATUS_ERROR}" "${M}" + STATUS_VARIABLES+=( "installing_Trutype_fonts='true'\n" ) + else + display_msg --logonly info "Skipping: Installing Trutype fonts - already installed" + fi + # Do the rest, even if we already did it in a previous installation, + # in case something in the directories changed. + + display_msg --log progress "Setting up modules and overlays." # These will get overwritten if the user has prior versions. cp -ar "${ALLSKY_REPO}/overlay" "${ALLSKY_CONFIG}" cp -ar "${ALLSKY_REPO}/modules" "${ALLSKY_CONFIG}" @@ -1899,30 +2203,23 @@ install_overlay() sudo mkdir -p "${ALLSKY_MODULE_LOCATION}/modules" sudo chown -R "${ALLSKY_OWNER}:${WEBSERVER_GROUP}" "${ALLSKY_MODULE_LOCATION}" sudo chmod -R 774 "${ALLSKY_MODULE_LOCATION}" - - # TODO: Remove in next release. Temporary fix to move modules and deal with - # pistatus and gps that moved to core allsky during testing of "dev" release. - if [[ -d /etc/allsky/modules ]]; then - sudo cp -a /etc/allsky/modules "${ALLSKY_MODULE_LOCATION}" - sudo rm -rf /etc/allsky - fi - sudo rm -f "${ALLSKY_MODULE_LOCATION}/modules/allsky_pistatus.py" - sudo rm -f "${ALLSKY_MODULE_LOCATION}/modules/allsky_script.py" - #TODO: End } #### check_if_buster() { + STATUS_VARIABLES+=("check_if_buster='true'\n") + if [[ ${OS} == "buster" ]]; then - MSG="This release runs best on the newer Bullseye operating system" + MSG="This release runs best on the Bullseye operating system" MSG="${MSG} that was released in November, 2021." MSG="${MSG}\nYou are running the older Buster operating system and we" MSG="${MSG} recommend doing a fresh install of Bullseye on a clean SD card." MSG="${MSG}\n\nDo you want to continue anyhow?" - if ! whiptail --title "${TITLE}" --yesno "${MSG}" 18 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then - exit_installation 0 + if ! whiptail --title "${TITLE}" --yesno --defaultno "${MSG}" 18 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + display_msg --logonly info "User running Buster and elected not to continue." + exit_installation 0 "${STATUS_NOT_CONTINUE}" "After Buster check." fi fi } @@ -1961,9 +2258,11 @@ display_image() # Replace the "installing" messaged with a "failed" one. exit_with_image() { + local RET="${1}" + local STATUS="${2}" + local MORE_STATUS="${2}" display_image "InstallationFailed" - #shellcheck disable=SC2086 - exit_installation ${1} + exit_installation "${RET}" "${STATUS}" "${MORE_STATUS}" } @@ -1973,7 +2272,7 @@ check_restored_settings() if [[ ${RESTORED_PRIOR_SETTINGS_FILE} == "true" && \ ${RESTORED_PRIOR_CONFIG_SH} == "true" && \ ${RESTORED_PRIOR_FTP_SH} == "true" ]]; then - # If we restored all the prior settings so no configuration is needed. + # If we restored all the prior settings no configuration is needed. if [[ ${WILL_REBOOT} == "true" ]]; then IMG="" # Removes existing image else @@ -1994,7 +2293,8 @@ check_restored_settings() MSG="Default files were created for:" [[ ${RESTORED_PRIOR_CONFIG_SH} == "false" ]] && MSG="${MSG}\n config.sh" [[ ${RESTORED_PRIOR_FTP_SH} == "false" ]] && MSG="${MSG}\n ftp-settings.sh" - MSG="${MSG}\n\nHowever, you must update them by going to the 'Editor' page in the WebUI after rebooting." + MSG="${MSG}\n\nHowever, you must update them by going to the" + MSG="${MSG} 'Editor' page in the WebUI after rebooting." whiptail --title "${TITLE}" --msgbox "${MSG}" 12 "${WT_WIDTH}" 3>&1 1>&2 2>&3 fi @@ -2002,12 +2302,39 @@ check_restored_settings() } +#### +# See if the new ZWO exposure algorithm should be used. +check_new_exposure_algorithm() +{ + local FIELD="experimentalExposure" + local NEW="$( settings ".${FIELD}" )" + [[ ${NEW} -eq 1 ]] && return + + MSG="There is a new auto-exposure algorithm for nighttime images that initial testing indicates" + MSG="${MSG} it creates better images at night and during the day-to-night transition." + MSG="${MSG}\n\nDo you want to use it?" + if whiptail --title "${TITLE}" --yesno "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + display_msg --logonly info "Enabling ${FIELD}." + update_json_file ".${FIELD}" 1 "${SETTINGS_FILE}" + + MSG="Please provide feedback on the new auto-exposure algorithm" + MSG="${MSG} by entering a Discussion item in GitHub." + MSG="${MSG}\nYou can disable it by changing 'New Exposure Algorithm' in the WebUI." + display_msg notice "${MSG}" + else + display_msg --logonly info "User elected NOT to use ${FIELD}." + fi + + STATUS_VARIABLES+=( "check_new_exposure_algorithm='true'\n" ) +} + + #### remind_old_version() { if [[ -n ${PRIOR_ALLSKY} ]]; then MSG="When you are sure everything is working with the new Allsky release," - MSG="${MSG} remove your old version in ${PRIOR_ALLSKY_DIR} to save disk space." + MSG="${MSG} remove your old version in '${PRIOR_ALLSKY_DIR}' to save disk space." whiptail --title "${TITLE}" --msgbox "${MSG}" 12 "${WT_WIDTH}" 3>&1 1>&2 2>&3 fi } @@ -2017,27 +2344,72 @@ remind_old_version() remind_run_check_allsky() { MSG="After you've configured Allsky, run:" - MSG="${MSG}\n cd ~/allsky; scripts/check_allsky.sh" - MSG="${MSG}\nto check for any issues. You can also run it whenever you make changes." + MSG="${MSG}\n\n cd ~/allsky; scripts/check_allsky.sh" + MSG="${MSG}\n\nto check for any issues. You can also run it whenever you make changes." whiptail --title "${TITLE}" --msgbox "${MSG}" 12 "${WT_WIDTH}" 3>&1 1>&2 2>&3 - display_msg --logonly info "${MSG}" + display_msg --logonly info "Displayed message about running 'check_allsky.sh'." +} + + +clear_status() +{ + rm -f "${STATUS_FILE}" } +# Update the status from the specified file. +# It's ok if the file doesn't exist. +update_status_from_temp_file() +{ + if [[ -s ${STATUS_FILE_TEMP} ]]; then + STATUS_VARIABLES+=( "$( < "${STATUS_FILE_TEMP}" )" ) + STATUS_VARIABLES+=("\n") + rm -f "${STATUS_FILE_TEMP}" + fi +} #### exit_installation() { [[ -z ${FUNCTION} ]] && display_msg "${LOG_TYPE}" info "\nENDING INSTALLATON AT $(date).\n" - local E="${1}" + local RET="${1}" + + # If STATUS_LINE is set, add that and all STATUS_VARIABLES to the status file. + local STATUS_CODE="${2}" + local MORE_STATUS="${3}" + if [[ -n ${STATUS_CODE} ]]; then + if [[ ${STATUS_CODE} == "${STATUS_CLEAR}" ]]; then + clear_status + else + [[ -n ${MORE_STATUS} ]] && MORE_STATUS="; MORE_STATUS='${MORE_STATUS}'" + echo -e "STATUS_INSTALLATION='${STATUS_CODE}'${MORE_STATUS}" > "${STATUS_FILE}" + update_status_from_temp_file + echo -e "${STATUS_VARIABLES[@]}" >> "${STATUS_FILE}" + + # If the user needs to reboot, save the current uptime-since + # so we can check it when Allsky starts. If it's the same value + # the user did not reboot. + # If the time is different the user rebooted. + if [[ ${STATUS_CODE} == "${STATUS_NO_FINISH_REBOOT}" || + ${STATUS_CODE} == "${STATUS_NO_REBOOT}" ]]; then + uptime --since > "${ALLSKY_REBOOT_NEEDED}" + fi + fi + fi + + # Don't exit for negative numbers. #shellcheck disable=SC2086 - [[ ${E} -ge 0 ]] && exit ${E} + [[ ${RET} -ge 0 ]] && exit ${RET} } +#### +handle_interrupts() +{ + display_msg --log info "\nGot interrupt - saving installation status, then exiting.\n" + exit_installation 1 "${STATUS_INT}" "Saving status." +} -####################### Main part of program - -OS="$(grep CODENAME /etc/os-release | cut -d= -f2)" # usually buster or bullseye +############################################## Main part of program ##### Calculate whiptail sizes calc_wt_size @@ -2062,24 +2434,10 @@ if [[ ${IN_TESTING} == "true" ]]; then MSG="${MSG}\nChanges from prior dev releases:" - X="/etc/allsky/modules" - if [[ -d ${X} ]]; then - MSG="${MSG}\n" - MSG="${MSG} * ${X} is no longer used." - MSG="${MSG} Move its contents to ${ALLSKY_MODULE_LOCATION} then 'sudo rmdir ${X}" - fi - - MSG="${MSG}\n * The allsky/tmp/extra directory moved to '${ALLSKY_EXTRA}'." - MSG="${MSG}\n YOU need to move any files to the new location and UPDATE YOUR SCRIPTS." + MSG="${MSG}\n * change 1" MSG="${MSG}\n" - MSG="${MSG}\n * The '${ALLSKY_CONFIG}/overlay/config/fields.json' file used to" - MSG="${MSG}\n contain both System fields and User fields (ones YOU created)." - MSG="${MSG}\n It now includes only System fields." - MSG="${MSG}\n After this installation please re-add any User fields via the" - MSG="${MSG}\n Variable Manager in the WebUI. Look in the old 'fields.json'" - MSG="${MSG}\n file for a list of your field and their attributes." - MSG="${MSG}\n Future updates will preserve your user fields." + MSG="${MSG}\n * change 2" MSG="${MSG}\n\nIf you agree, enter: yes" A=$(whiptail --title "*** MESSAGE FOR TESTERS ***" --inputbox "${MSG}" 26 "${WT_WIDTH}" 3>&1 1>&2 2>&3) @@ -2126,11 +2484,11 @@ TESTING="${TESTING}" # xxx keeps shellcheck quiet shift done + if [[ -n ${FUNCTION} ]]; then # Don't log when a single function is executed. DISPLAY_MSG_LOG="" else - ##### Log files write to ${ALLSKY_CONFIG}, which doesn't exist yet, so create it. mkdir -p "${ALLSKY_INSTALLATION_LOGS}" display_msg "${LOG_TYPE}" info "STARTING INSTALLATON AT $(date).\n" @@ -2139,7 +2497,88 @@ fi [[ ${HELP} == "true" ]] && usage_and_exit 0 [[ ${OK} == "false" ]] && usage_and_exit 1 -##### Does a prior Allsky exist? If so, set PRIOR_ALLSKY +trap "handle_interrupts" SIGTERM SIGINT + +# See if we should skip some steps. +# When most function are called they add a variable with the function's name set to "true". +if [[ -z ${FUNCTION} && -s ${STATUS_FILE} ]]; then + # Initially just get the status. + # After that we may clear the file or get all the variables. + eval "$( grep STATUS_INSTALLATION "${STATUS_FILE}" )" + + if [[ ${STATUS_INSTALLATION} == "${STATUS_OK}" ]]; then + MSG="The last installation completed successfully." + MSG="${MSG}\n\nDo you want to re-install from the beginning?" + MSG="${MSG}\n\nSelecting will exit the installation without making any changes." + if whiptail --title "${TITLE}" --yesno "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + display_msg --log progress "Re-starting installation after successful install." + clear_status + else + display_msg --log progress "Not continuing after prior successful installation." + exit_installation 0 "" + fi + elif [[ ${STATUS_INSTALLATION} == "${STATUS_NO_FINISH_REBOOT}" ]]; then + MSG="The installation completed successfully but the following needs to happen" + MSG="${MSG} before Allsky is ready to run:" + MSG2="\n\n 1. Verify your settings in the WebUi's 'Allsky Settings' page." + MSG2="${MSG2}\n 2. Reboot the Pi." + MSG3="\n\nHave you already performed those steps?" + if whiptail --title "${TITLE}" --yesno "${MSG}${MSG2}${MSG3}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + MSG="\nCongratulations, you successfully installed Allsky version ${ALLSKY_VERSION}!" + MSG="${MSG}\nAllsky is starting. Look in the 'Live View' page of the WebUI to ensure" + MSG="${MSG}\nimages are being taken.\n" + display_msg --log progress "${MSG}" + sudo systemctl start allsky + + # Update status + sed -i \ + -e "s/${STATUS_NO_FINISH_REBOOT}/${STATUS_OK}/" \ + -e "s/MORE_STATUS.*//" \ + "${STATUS_FILE}" + else + display_msg --log info "\nPlease perform the following steps:${MSG2}\n" + fi + exit_installation 0 "" "" + else + MSG="You have already begun the installation." + MSG="${MSG}\n\nThe last status was: ${STATUS_INSTALLATION}${MORE_STATUS}" + MSG="${MSG}\n\nDo you want to continue where you left off?" + if whiptail --title "${TITLE}" --yesno "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + MSG="Continuing installation. Steps already performed will be skipped." + MSG="${MSG}\nThe last status was: ${STATUS_INSTALLATION}${MORE_STATUS}" + display_msg --log progress "${MSG}" + + #shellcheck disable=SC1090 # file doesn't exist in GitHub + source "${STATUS_FILE}" || exit 1 + # Put all but the status variable in the list so we save them next time. + STATUS_VARIABLES=( "$( grep -v STATUS_INSTALLATION "${STATUS_FILE}" )" ) + STATUS_VARIABLES+=("\n#### Prior variables above, new below.\n") + + # If returning from a reboot for local, + # prompt for locale again to make sure it's there and still what they want. + if [[ ${STATUS_INSTALLATION} == "${STATUS_LOCALE_REBOOT}" ]]; then + unset get_desired_locale # forces a re-prompt + unset CURRENT_LOCALE # It will get re-calculated + fi + + else + MSG="Do you want to restart the installation from the beginning?" + MSG="${MSG}\n\nSelecting will exit the installation without making any changes." + if whiptail --title "${TITLE}" --yesno "${MSG}" 15 "${WT_WIDTH}" 3>&1 1>&2 2>&3; then + display_msg --log progress "Restarting installation." + else + display_msg --log progress "Not continuing after prior partial installation." + exit_installation 0 "" + fi + fi + fi +fi + +##### Display a message to Buster users. +[[ ${check_if_buster} != "true" && -z ${FUNCTION} ]] && check_if_buster + +##### Does a prior Allsky exist? If so, set PRIOR_ALLSKY and other PRIOR_* variables. +# Re-run every time in case the directory was removed. does_prior_Allsky_exist ##### Display the welcome header @@ -2148,14 +2587,18 @@ does_prior_Allsky_exist ##### Stop Allsky stop_allsky +##### Determine what camera(s) are connected +# Re-run every time in case a camera was connected or disconnected. +get_connected_cameras + ##### Get branch -get_this_branch +[[ ${get_this_branch} != "true" ]] && get_this_branch ##### Handle updates [[ ${UPDATE} == "true" ]] && do_update # does not return ##### See if there's an old WebUI -does_old_WebUI_location_exist +[[ ${does_old_WebUI_location_exist} != "true" ]] && does_old_WebUI_location_exist ##### Executes the specified function, if any, and exits. if [[ -n ${FUNCTION} ]]; then @@ -2169,78 +2612,87 @@ display_image "InstallationInProgress" # Do as much of the prompting up front, then do the long-running work, then prompt at the end. ##### Prompt to use prior Allsky -prompt_for_prior_Allsky +[[ ${prompt_for_prior_Allsky} != "true" ]] && prompt_for_prior_Allsky ##### Get locale (prompt if needed). May not return. -get_locale +[[ ${get_desired_locale} != "true" ]] && get_desired_locale -##### Determine the camera type -select_camera_type +##### Prompt for the camera type +[[ ${select_camera_type} != "true" ]] && select_camera_type ##### Get the new host name -prompt_for_hostname +[[ ${prompt_for_hostname} != "true" ]] && prompt_for_hostname ##### Check for sufficient swap space -check_swap +[[ ${check_swap} != "true" ]] && check_swap ##### Optionally make ${ALLSKY_TMP} a memory filesystem -check_tmp +[[ ${check_tmp} != "true" ]] && check_tmp -MSG="\nThe following steps can take up to 1 - 3 HOURS depending on the speed of your Pi" +MSG="The following steps can take up to an hour depending on the speed of your Pi" MSG="${MSG}\nand how many of the necessary dependencies are already installed." -display_msg info "${MSG}" - MSG="${MSG}\nYou will see progress messages throughout the process." -MSG="${MSG}\nAt the end you will be prompted again for additional steps.\n" -whiptail --title "${TITLE}" --msgbox "${MSG}" 12 "${WT_WIDTH}" 3>&1 1>&2 2>&3 +MSG="${MSG}\nAt the end you will be prompted again for additional steps." +display_msg notice "${MSG}" ##### Install web server # This must come BEFORE save_camera_capabilities, since it installs php. -install_webserver +[[ ${install_webserver} != "true" ]] && install_webserver ##### Install dependencies, then compile and install Allsky software # This will create the "config" directory and put default files in it. -install_dependencies_etc || exit_with_image 1 +[[ ${install_dependencies_etc} != "true" ]] && install_dependencies_etc ##### Create the file that defines the WebUI variables. -create_webui_defines +[[ ${create_webui_defines} != "true" ]] && create_webui_defines -##### Create the camera type-model-specific "options" file +##### Create the camera type/model-specific "options" file # This should come after the steps above that create ${ALLSKY_CONFIG}. -save_camera_capabilities "false" || exit_with_image 1 # prompts on error only +if [[ ${save_camera_capabilities} != "true" ]]; then + save_camera_capabilities "false" # prompts on error only + [[ $? -ne 0 ]] && exit_with_image 1 "${STATUS_ERROR}" "save_camera_capabilities failed." +fi ##### Set locale. May reboot instead of returning. -set_locale +[[ ${set_locale} != "true" ]] && set_locale ##### Create the Allsky log files +# Re-run every time in case permissions changed. create_allsky_logs ##### install the overlay and modules system install_overlay ##### Check for, and handle any prior Allsky Website -handle_prior_website +[[ ${handle_prior_website} != "true" ]] && handle_prior_website ##### Restore prior files if needed -restore_prior_files # prompts if prior Allsky exists +[[ ${restore_prior_files} != "true" ]] && restore_prior_files # prompts if prior Allsky exists ##### Update config.sh -update_config_sh +[[ ${update_config_sh} != "true" ]] && update_config_sh ##### Set permissions. Want this at the end so we make sure we get all files. +# Re-run every time in case permissions changed. set_permissions ##### Check if there's an old WebUI and let the user know it's no longer used. +# Re-run every time to remind them if there's still an old location. check_old_WebUI_location # prompt if prior old-style WebUI ##### See if we should reboot when installation is done. -ask_reboot "full" # prompts +[[ ${REBOOT_NEEDED} == "true" ]] && ask_reboot "full" # prompts ##### Display any necessary messaged about restored / not restored settings +# Re-run every time to possibly remind them to update their settings. check_restored_settings +##### If using ZWO, prompt if the New Exposure Algorithm should be used. +# TODO: remove check_new_exposure_algorithm() when it's the default. +[[ ${CAMERA_TYPE} == "ZWO" && ${check_new_exposure_algorithm} != "true" ]] && check_new_exposure_algorithm + ##### Let the user know to run check_allsky.sh. remind_run_check_allsky @@ -2250,6 +2702,10 @@ remind_old_version ######## All done -[[ ${WILL_REBOOT} == "true" ]] && do_reboot # does not return +[[ ${WILL_REBOOT} == "true" ]] && do_reboot "${STATUS_FINISH_REBOOT}" "" # does not return -exit_installation 0 +if [[ ${REBOOT_NEEDED} == "true" ]]; then + exit_installation 0 "${STATUS_NO_FINISH_REBOOT}" "" +else + exit_installation 0 "${STATUS_OK}" "" +fi diff --git a/scripts/check_allsky.sh b/scripts/check_allsky.sh index 71eb336d5..595a00303 100755 --- a/scripts/check_allsky.sh +++ b/scripts/check_allsky.sh @@ -28,7 +28,7 @@ usage_and_exit() # Don't show the "--newer", "--no-check", or "--force-check" options since users # should never use them. echo - echo -e "${C}Usage: ${ME} [--help] [--debug]${NC}" + echo -e "${C}Usage: ${ME} [--help] [--debug] [--no-check]${NC}" echo echo "'--help' displays this message and exits." echo @@ -76,8 +76,7 @@ source "${ALLSKY_CONFIG}/config.sh" || exit ${ALLSKY_ERROR_STOP} source "${ALLSKY_CONFIG}/ftp-settings.sh" || exit ${ALLSKY_ERROR_STOP} PROTOCOL="${PROTOCOL,,}" # set to lowercase to make comparing easier - -BRANCH="$( get_branch "${ALLSKY_BRANCH_FILE}" )" +BRANCH="$( get_branch "" )" [[ -z ${BRANCH} ]] && BRANCH="${GITHUB_MAIN_BRANCH}" [[ ${DEBUG} == "true" ]] && echo "DEBUG: using '${BRANCH}' branch." @@ -152,7 +151,7 @@ function heading() fi } -# Determine if the specified value is a number +# Determine if the specified value is a number. function is_number() { local VALUE="${1}" @@ -168,6 +167,18 @@ function is_number() fi } +# Return the min of two numbers. +function min() { + local ONE="${1}" + local TWO="${2}" + if [[ ${ONE} -lt ${TWO} ]]; then + echo "${ONE}" + else + echo "${TWO}" + fi +} + +# =================================================== CHECKING FUNCTIONS # The various upload protocols need different variables defined. # For the specified protocol, make sure the specified variable is defined. @@ -196,20 +207,61 @@ function check_exists() { fi } + + +DAYDELAY_MS=$(settings .daydelay) || echo "Problem getting .daydelay" +NIGHTDELAY_MS=$(settings .nightdelay) || echo "Problem getting .nightdelay" + + # Use min() for worst case. +MIN_DELAY_MS=$( min "${DAYDELAY_MS}" "${NIGHTDELAY_MS}" ) + # This is typically the max daytime exposure, which is shorter than nighttime so use it. +MIN_EXPOSURE_MS=250 + # Minimum total time spent on each image +MIN_IMAGE_TIME_MS=$((MIN_EXPOSURE_MS + MIN_DELAY_MS)) + +##### Check if the delay is so short it's likely to cause problems. +function check_delay() +{ +# TODO: use the module average flow times for day and night + + # With the legacy overlay method it might take up to a couple seconds to save an image. + # With the module method it can take up to 5 seconds. + local OVERLAY_METHOD=$(settings .overlayMethod) || echo "Problem getting .overlayMethod." >&2 + if [[ ${OVERLAY_METHOD} -eq 1 ]]; then + MAX_TIME_TO_SAVE_MS=5000 + else + MAX_TIME_TO_SAVE_MS=2000 + fi + if [[ ${MAX_TIME_TO_SAVE_MS} -gt ${MIN_IMAGE_TIME_MS} ]]; then + heading "Warnings" + echo "The minimum delay of ${MIN_DELAY_MS} may be too short" + echo "given the maximum expected time to save and process" + echo "an image (${MAX_TIME_TO_SAVE_MS} ms)." + echo "A new image may appear before the prior one has finished processing." + echo "Consider increasing your delay." + fi +} + +# +# ====================================================== MAIN PART OF PROGRAM +# + # Variables used below. -TAKING_DARKS="$(settings .takeDarkFrames)" -WIDTH="$(settings .width)" # per the WebUI, usually 0 -HEIGHT="$(settings .height)" -SENSOR_WIDTH="$(settings .sensorWidth "${CC_FILE}")" # physical sensor size -SENSOR_HEIGHT="$(settings .sensorHeight "${CC_FILE}")" -TAKE="$(settings .takeDaytimeImages)" -SAVE="$(settings .saveDaytimeImages)" -ANGLE="$(settings .angle)" -LATITUDE="$(settings .latitude)" -LONGITUDE="$(settings .longitude)" +TAKING_DARKS="$(settings .takeDarkFrames)" || echo "Problem getting .takeDarkFrames." >&2 +# per the WebUI, width and height are usually 0 +WIDTH="$(settings .width)" || echo "Problem getting .width." >&2 +HEIGHT="$(settings .height)" || echo "Problem getting .height." >&2 +# physical sensor size +SENSOR_WIDTH="$(settings .sensorWidth "${CC_FILE}")" || echo "Problem getting .sensorWidth." >&2 +SENSOR_HEIGHT="$(settings .sensorHeight "${CC_FILE}")" || echo "Problem getting .sensorHeight." >&2 +TAKE="$(settings .takeDaytimeImages)" || echo "Problem getting .takeDaytimeImages." >&2 +SAVE="$(settings .saveDaytimeImages)" || echo "Problem getting .saveDaytimeImages." >&2 +ANGLE="$(settings .angle)" || echo "Problem getting .angle" >&2 +LATITUDE="$(settings .latitude)" || echo "Problem getting .latitude." >&2 +LONGITUDE="$(settings .longitude)" || echo "Problem getting .longitude" >&2 # shellcheck disable=SC2034 -LOCALE="$(settings .locale)" -USING_DARKS="$(settings .useDarkFrames)" +LOCALE="$(settings .locale)" || echo "Problem getting .locale" >&2 +USING_DARKS="$(settings .useDarkFrames)" || echo "Problem getting .useDarkFrames" >&2 WEBSITES="$(whatWebsites)" # ====================================================================== @@ -220,7 +272,7 @@ WEBSITES="$(whatWebsites)" if [[ ${TAKING_DARKS} -eq 1 ]]; then heading "Information" echo "'Take Dark Frames' is set." - echo "Unset if you no longer wish to take dark frames." + echo "Unset when you are done taking dark frames." fi if [[ ${KEEP_SEQUENCE} == "true" ]]; then @@ -275,14 +327,28 @@ if [[ ${CROP_IMAGE} == "true" && ${SENSOR_WIDTH} == "${CROP_WIDTH}" && ${SENSOR_ echo "Check CROP_IMAGE, CROP_WIDTH (${CROP_WIDTH}), and CROP_HEIGHT (${CROP_HEIGHT})." fi +LAST_CHANGED="$( settings ".lastChanged" )" || echo "Problem getting .lastChanged" >&2 +if [[ ${LAST_CHANGED} == "" || ${LAST_CHANGED} == "null" ]]; then + heading "Information" + echo "Allsky needs to be configured before it will run." + echo "See the 'Allsky Settings' page in the WebUI." +fi + +if reboot_needed ; then + heading "Information" + echo "The Pi needs to be rebooted before Allsky will start." +fi # ====================================================================== # ================= Check for warning items. # These are wrong and won't stop Allsky from running, but # may break part of Allsky, e.g., uploads may not work. +##### Check if the delay is so short it's likely to cause problems. +check_delay -# Check if timelapse size is "too big" and will likely cause an error. + +##### Check if timelapse size is "too big" and will likely cause an error. # This is normally only an issue with the libx264 video codec which has a dimension limit # that we put in PIXEL_LIMIT if [[ ${VCODEC} == "libx264" ]]; then @@ -358,6 +424,7 @@ if [[ ${TIMELAPSE} == "false" && ${UPLOAD_VIDEO} == "true" ]]; then echo "Timelapse videos are not being created (TIMELAPSE='false') but UPLOAD_VIDEO='true'" fi + if [[ ${TIMELAPSE_MINI_IMAGES} -gt 0 ]]; then # See if there's likely to be a problem with mini timelapse creations @@ -367,60 +434,28 @@ if [[ ${TIMELAPSE_MINI_IMAGES} -gt 0 ]]; then # 2. Frequency: how often mini timelapse are created (i.e., after how many images) # 3. NumImages: how many images are used (the more the longer processing takes) # 4. the speed of the Pi - this is the biggest unknown - function min() { # return the min of two numbers - ONE=$(settings "${1}") - TWO=$(settings "${2}") - if [[ ${ONE} -lt ${TWO} ]]; then - echo "${ONE}" - else - echo "${TWO}" - fi - } function get_exposure() { # return the time spent on one image, prior to delay - TIME="${1}" - if [[ $(settings ".${TIME}autoexposure") ]]; then - settings ".${TIME}maxautoexposure" + local TIME="${1}" + if [[ $(settings ".${TIME}autoexposure") -eq 1 ]]; then + settings ".${TIME}maxautoexposure" || echo "Problem getting .${TIME}maxautoexposure." >&2 else - settings ".${TIME}exposure" + settings ".${TIME}exposure" || echo "Problem getting .${TIME}exposure." >&2 fi } - # Use min() for worst case. - # Convert to seconds ( / 1000) to make logic easier. - MIN_DELAY=$(min .daydelay .nightdelay) - MIN_DELAY=$((MIN_DELAY / 1000)) -# TODO: remove the commented out assigments after we know this works. -#MIN_DELAY=1 -#TIMELAPSE_MINI_FREQUENCY=10 - -# TODO: Hard-code the MIN_EXPOSURE below instead of these lines: -#x CONSISTENT_DELAYS=$(settings .consistentDelays) -#x if [[ ${CONSISTENT_DELAYS} -eq 1 ]]; then -#x MIN_EXPOSURE=$(min .daymaxautoexposure .nightmaxautoexposure) -#x else -#x MIN_EXPOSURE=$(min "$(get_exposure "day")" "$(get_exposure "night")") -#x fi -#x MIN_EXPOSURE=$((MIN_EXPOSURE / 1000)) - - # This is typically the max daytime exposure, which is shorter than nighttime. - # so use it. - MIN_EXPOSURE=0.25 - - # Minimum total time spent on each image - MIN_IMAGE_TIME=$(echo "${MIN_EXPOSURE} + ${MIN_DELAY}" | bc -l) - # Minimum total time between start of timelapse creations. - MIN_TIME_BETWEEN_TIMELAPSE=$(echo "scale=0; ${TIMELAPSE_MINI_FREQUENCY} * ${MIN_IMAGE_TIME}" | bc -l) - MIN_TIME_BETWEEN_TIMELAPSE=${MIN_TIME_BETWEEN_TIMELAPSE/.*/} - -#echo "CONSISTENT_DELAYS=$CONSISTENT_DELAYS" -#echo "MIN_DELAY=$MIN_DELAY" -#echo "MIN_EXPOSURE=$MIN_EXPOSURE" -#echo "MIN_IMAGE_TIME=$MIN_IMAGE_TIME" -#echo "MIN_TIME_BETWEEN_TIMELAPSE=$MIN_TIME_BETWEEN_TIMELAPSE" -#TIMELAPSE_MINI_IMAGES=120 -#echo "TIMELAPSE_MINI_IMAGES=$TIMELAPSE_MINI_IMAGES" -#echo "CAMERA_TYPE=$CAMERA_TYPE" + MIN_IMAGE_TIME_SEC=$(( MIN_IMAGE_TIME_MS / 1000)) + MIN_TIME_BETWEEN_TIMELAPSE_SEC=$(echo "scale=0; ${TIMELAPSE_MINI_FREQUENCY} * ${MIN_IMAGE_TIME_SEC}" | bc -l) + MIN_TIME_BETWEEN_TIMELAPSE_SEC=${MIN_TIME_BETWEEN_TIMELAPSE_SEC/.*/} + +if false; then # for testing + echo "CONSISTENT_DELAYS=${CONSISTENT_DELAYS}" + echo "MIN_IMAGE_TIME_SEC=${MIN_IMAGE_TIME_SEC}" + echo "MIN_TIME_BETWEEN_TIMELAPSE_SEC=${MIN_TIME_BETWEEN_TIMELAPSE_SEC}" + echo "TIMELAPSE_MINI_IMAGES=${TIMELAPSE_MINI_IMAGES}" + echo "CAMERA_TYPE=${CAMERA_TYPE}" + TIMELAPSE_MINI_IMAGES=120 +fi # On a Pi 4, creating a 50 image timelapse takes # - a few seconds on a small ZWO camera @@ -432,14 +467,13 @@ if [[ ${TIMELAPSE_MINI_IMAGES} -gt 0 ]]; then S=60 fi EXPECTED_TIME=$(echo "scale=0; (${TIMELAPSE_MINI_IMAGES} / 50) * ${S}" | bc -l) -#echo "EXPECTED_TIME=$EXPECTED_TIME" - if [[ ${EXPECTED_TIME} -gt ${MIN_TIME_BETWEEN_TIMELAPSE} ]]; then + if [[ ${EXPECTED_TIME} -gt ${MIN_TIME_BETWEEN_TIMELAPSE_SEC} ]]; then heading "Warnings" echo "Your mini timelapse settings may cause multiple timelapse to be created simultaneously." echo "Consider increasing DELAY between pictures, increasing TIMELAPSE_MINI_FREQUENCY," echo "decrease TIMELAPSE_MINI_IMAGES, or a combination of those changes." echo "Expected time to create a mini timelapse on a Pi 4 is ${EXPECTED_TIME} seconds" - echo "but with your settings one will be created as short as every ${MIN_TIME_BETWEEN_TIMELAPSE} seconds." + echo "but with your settings one will be created as short as every ${MIN_TIME_BETWEEN_TIMELAPSE_SEC} seconds." fi fi @@ -463,17 +497,6 @@ if [[ ${STARTRAILS} == "false" && ${UPLOAD_STARTRAILS} == "true" ]]; then echo "Startrails are not being created (STARTRAILS='false') but UPLOAD_STARTRAILS='true'" fi - -if [[ ${RESIZE_UPLOADS} == "true" && ${IMG_UPLOAD} == "false" ]]; then - heading "Warnings" - echo "RESIZE_UPLOADS is 'true' but you aren't uploading images (IMG_UPLOAD='false')." -fi - -if [[ ${TAKE} -eq 0 && ${SAVE} -eq 1 ]]; then - heading "Warnings" - echo "'Daytime Capture' is off but 'Daytime Save' is on in the WebUI." -fi - if [[ ${BRIGHTNESS_THRESHOLD} == "0.0" ]]; then heading "Warnings" echo "BRIGHTNESS_THRESHOLD is 0.0 which means ALL images will be IGNORED when creating startrails." @@ -482,9 +505,11 @@ elif [[ ${BRIGHTNESS_THRESHOLD} == "1.0" ]]; then echo "BRIGHTNESS_THRESHOLD is 1.0 which means ALL images will be USED when creating startrails, even daytime images." fi -if [[ -f ${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE} && (${PROTOCOL} == "" || ${PROTOCOL} == "local") ]]; then +##### Images + +if [[ ${TAKE} -eq 0 && ${SAVE} -eq 1 ]]; then heading "Warnings" - echo "A remote Allsky Website configuration file was found but PROTOCOL doesn't support uploading files." + echo "'Daytime Capture' is off but 'Daytime Save' is on in the WebUI." fi if [[ ${REMOVE_BAD_IMAGES} != "true" ]]; then @@ -493,6 +518,11 @@ if [[ ${REMOVE_BAD_IMAGES} != "true" ]]; then echo We HIGHLY recommend setting it to 'true' unless you are debugging issues. fi +##### Uploads +if [[ ${RESIZE_UPLOADS} == "true" && ${IMG_UPLOAD} == "false" ]]; then + heading "Warnings" + echo "RESIZE_UPLOADS is 'true' but you aren't uploading images (IMG_UPLOAD='false')." +fi case "${PROTOCOL}" in "" | local) # Nothing needed for these @@ -502,6 +532,10 @@ case "${PROTOCOL}" in check_PROTOCOL "${PROTOCOL}" "REMOTE_HOST" check_PROTOCOL "${PROTOCOL}" "REMOTE_USER" check_PROTOCOL "${PROTOCOL}" "REMOTE_PASSWORD" + if [[ ${PROTOCOL} == "ftp" ]]; then + heading "Warnings" + echo "PROTOCOL set to insecure 'ftp'. Try to use 'ftps' or 'sftp' instead." + fi ;; scp) @@ -541,34 +575,17 @@ if [[ -n ${REMOTE_PORT} ]] && ! is_number "${REMOTE_PORT}" ; then echo "Uploads will not work until this is corrected." fi -# If these variables are set, their corresponding directory should exist. +##### If these variables are set, their corresponding directory should exist. check_exists "WEB_IMAGE_DIR" check_exists "WEB_VIDEOS_DIR" check_exists "WEB_KEOGRAM_DIR" check_exists "WEB_STARTRAILS_DIR" check_exists "UHUBCTL_PATH" -# Check for Allsky Website-related anomolies. -if [[ ${WEBSITES} != "none" ]]; then - if [[ ${IMG_UPLOAD} == "false" ]]; then - heading "Warnings" - echo "You have an Allsky Website but no images are being uploaded to it (IMG_UPLOAD=false)." - fi - if [[ ${TIMELAPSE} == "true" && ${UPLOAD_VIDEO} == "false" ]]; then - heading "Warnings" - echo "You have an Allsky Website and timelapse videos are being created (TIMELAPSE=true)," - echo "but they are not being uploaded (UPLOAD_VIDEO=false)." - fi - if [[ ${KEOGRAM} == "true" && ${UPLOAD_KEOGRAM} == "false" ]]; then - heading "Warnings" - echo "You have an Allsky Website and keograms are being created (KEOGRAM=true)," - echo "but they are not being uploaded (UPLOAD_KEOGRAM=false)." - fi - if [[ ${STARTRAILS} == "true" && ${UPLOAD_STARTRAILS} == "false" ]]; then - heading "Warnings" - echo "You have an Allsky Website and startrails are being created (STARTRAILS=true)," - echo "but they are not being uploaded (UPLOAD_STARTRAILS=false)." - fi +##### Check for Allsky Website-related issues. +if [[ -f ${ALLSKY_REMOTE_WEBSITE_CONFIGURATION_FILE} && (${PROTOCOL} == "" || ${PROTOCOL} == "local") ]]; then + heading "Warnings" + echo "A remote Allsky Website configuration file was found but PROTOCOL doesn't support uploading files." fi @@ -577,12 +594,19 @@ fi # ================= Check for error items. # These are wrong and will likely keep Allsky from running. +##### Make sure it's a know camera type. if [[ ${CAMERA_TYPE} != "ZWO" && ${CAMERA_TYPE} != "RPi" ]]; then heading "Errors" echo "INTERNAL ERROR: CAMERA_TYPE (${CAMERA_TYPE}) not valid." fi -# Make sure these booleans have boolean values, or are blank. +##### Make sure the settings file is properly linked. +if ! MSG="$( check_settings_link "${SETTINGS_FILE}" )" ; then + heading "Errors" + echo -e "${MSG}" +fi + +##### Make sure these booleans have boolean values, or are blank. for i in IMG_UPLOAD IMG_UPLOAD_ORIGINAL_NAME IMG_RESIZE CROP_IMAGE AUTO_STRETCH \ RESIZE_UPLOADS IMG_CREATE_THUMBNAILS REMOVE_BAD_IMAGES TIMELAPSE UPLOAD_VIDEO \ TIMELAPSE_UPLOAD_THUMBNAIL TIMELAPSE_MINI_FORCE_CREATION TIMELAPSE_MINI_UPLOAD_VIDEO \ @@ -595,8 +619,8 @@ do fi done -# Check that all required settings are set. -# All others are optional. +##### Check that all required settings are set. All others are optional. +# TODO: determine from options.json file which are required. for i in ANGLE LATITUDE LONGITUDE LOCALE do if [[ -z ${!i} || ${!i} == "null" ]]; then @@ -605,7 +629,7 @@ do fi done -# Check that the required settings' values are valid. +##### Check that the required settings' values are valid. if [[ -n ${ANGLE} ]] && ! is_number "${ANGLE}" ; then heading "Errors" echo "ANGLE (${ANGLE}) must be a number." @@ -623,7 +647,7 @@ if [[ -n ${LONGITUDE} ]]; then fi fi -# Check dark frames +##### Check dark frames if [[ ${USING_DARKS} -eq 1 ]]; then if [[ ! -d ${ALLSKY_DARKS} ]]; then heading "Errors" @@ -638,7 +662,7 @@ if [[ ${USING_DARKS} -eq 1 ]]; then fi fi -# Check for valid numbers. +##### Check for valid numbers. if ! is_number "${IMG_UPLOAD_FREQUENCY}" || [[ ${IMG_UPLOAD_FREQUENCY} -le 0 ]]; then heading "Errors" echo "IMG_UPLOAD_FREQUENCY (${IMG_UPLOAD_FREQUENCY}) must be 1 or greater." @@ -679,7 +703,7 @@ if [[ ${REMOVE_BAD_IMAGES} == "true" ]]; then fi fi -# If images are being resized or cropped, +##### If images are being resized or cropped, # make sure the resized/cropped image is fully within the sensor image. HAS_PIXEL_ERROR="false" if [[ ${IMG_RESIZE} == "true" ]]; then diff --git a/scripts/copy_notification_image.sh b/scripts/copy_notification_image.sh index e56d0a130..dc999407b 100755 --- a/scripts/copy_notification_image.sh +++ b/scripts/copy_notification_image.sh @@ -98,7 +98,7 @@ if [[ ${NOTIFICATION_TYPE} != "custom" && -f ${ALLSKY_NOTIFICATION_LOG} && ${EXP NOW=$(date +'%Y-%m-%d %H:%M:%S') RESULTS="$(find "${ALLSKY_NOTIFICATION_LOG}" -newermt "${NOW}" -print)" if [[ -n ${RESULTS} ]]; then # the file is in the future - if [[ ${ALLSKY_DEBUG_LEVEL} -ge 3 ]]; then + if [[ ${ALLSKY_DEBUG_LEVEL} -ge 4 ]]; then # File contains: Notification_type,expires_in_seconds,expiration_time RECENT_NOTIFICATION=$(tail -1 "${ALLSKY_NOTIFICATION_LOG}") RECENT_TYPE=${RECENT_NOTIFICATION%%,*} @@ -195,7 +195,7 @@ if [[ ${IMG_UPLOAD} == "true" ]]; then # We're actually uploading ${UPLOAD_FILE}, but show ${NOTIFICATION_FILE} in the message since it's more descriptive. # If an existing notification is being uploaded, wait for it to finish then upload this one. - if [[ ${ALLSKY_DEBUG_LEVEL} -ge 2 ]]; then + if [[ ${ALLSKY_DEBUG_LEVEL} -ge 4 ]]; then echo -e "${ME}: Uploading $(basename "${NOTIFICATION_FILE}")" fi "${ALLSKY_SCRIPTS}/upload.sh" --wait --silent \ diff --git a/scripts/darkCapture.sh b/scripts/darkCapture.sh index a08f6c6ee..f3a7fa906 100755 --- a/scripts/darkCapture.sh +++ b/scripts/darkCapture.sh @@ -1,7 +1,7 @@ #!/bin/bash # This file is "source"d into another. -# "${CURRENT_IMAGE}" is the name of the current image we're working on and is passed to us. +# "${CURRENT_IMAGE}" is the full pathname of the current image we're working on and is passed to us. ME2="$(basename "${BASH_SOURCE[0]}")" # Make sure the input file exists; if not, something major is wrong so exit. @@ -14,49 +14,29 @@ if [[ ! -f ${CURRENT_IMAGE} ]]; then exit 2 fi -# ${AS_TEMPERATURE_C} is passed to us by saveImage.sh, but may be null. -# If ${AS_TEMPERATURE_C} is set, use it as the temperature, otherwise read the ${TEMPERATURE_FILE}. -# If the ${TEMPERATURE_FILE} file doesn't exist, set the temperature to "n/a". +# The extension on $CURRENT_IMAGE may not be $EXTENSION. +DARK_EXTENSION="${CURRENT_IMAGE##*.}" + +DARKS_DIR="${ALLSKY_DARKS}" +mkdir -p "${DARKS_DIR}" if [[ -z ${AS_TEMPERATURE_C} ]]; then - TEMPERATURE_FILE="${ALLSKY_TMP}/temperature.txt" - if [[ -s ${TEMPERATURE_FILE} ]]; then # -s so we don't use an empty file - AS_TEMPERATURE_C=$( < "${TEMPERATURE_FILE}") - else - AS_TEMPERATURE_C="n/a" - fi + # The camera doesn't support temperature so we'll keep overwriting the file until + # AS_TEMPERATURE_C is set. + # This allows users to continually look for a new dark file and rename it manually. + MOVE_TO_FILE="${DARKS_DIR}/$(basename "${CURRENT_IMAGE}")" +else + MOVE_TO_FILE="${DARKS_DIR}/${AS_TEMPERATURE_C}.${DARK_EXTENSION}" fi - -if [[ $(settings ".takeDarkFrames") -eq 1 ]]; then - # The extension on $CURRENT_IMAGE may not be $EXTENSION. - DARK_EXTENSION="${CURRENT_IMAGE##*.}" - - DARKS_DIR="${ALLSKY_DARKS}" - mkdir -p "${DARKS_DIR}" - # If the camera doesn't support temperature, we will keep overwriting the file until - # the user creates a temperature.txt file. - if [[ ${AS_TEMPERATURE_C} == "n/a" ]]; then - MOVE_TO_FILE="${DARKS_DIR}/$(basename "${CURRENT_IMAGE}")" - else - MOVE_TO_FILE="${DARKS_DIR}/${AS_TEMPERATURE_C}.${DARK_EXTENSION}" - fi - mv "${CURRENT_IMAGE}" "${MOVE_TO_FILE}" - - # If the user has notification images on, the current image says we're taking dark frames, - # so don't overwrite it. - # xxxx It's possible some people will want to see the dark frame even if notification images - # are being used - may need to make it optional to see the dark frame. - USE_NOTIFICATION_IMAGES=$(settings ".notificationimages") - if [[ ${USE_NOTIFICATION_IMAGES} -eq 0 ]]; then - # Go ahead and let the web sites see the dark frame to see if it's working. - # We're copying back the file we just moved, but the assumption is few people - # will want to see the dark frames on the web. - - # xxxx TODO: don't use $FULL_FILENAME since that assumes $EXTENSION is the same as - # $DARK_EXTENSION. If we start saving darks as .png files the extensions will be - # different and we'll need to run "convert" to make the dark a .jpg file to - # be displayed on the web. - cp "${MOVE_TO_FILE}" "$(dirname "${CURRENT_IMAGE}")/${FULL_FILENAME}" - fi - - exit 0 # exit so the calling script exits and doesn't try to process the file. +mv "${CURRENT_IMAGE}" "${MOVE_TO_FILE}" || exit 3 + +# If the user has notification images on, the current image says "Taking dark frames", +# so don't overwrite it. +# If notification images are off, let the user see the dark from to know it's working. +# Some people may want to see the dark frame even if notification images +# are being used, but no one's askef for that feature so don't worry about it. + +if [[ $(settings ".notificationimages") -eq 0 ]]; then + # We're copying back the file we just moved, but the assumption is few people + # will want to see the dark frames so the performance hit is + cp "${MOVE_TO_FILE}" "${ALLSKY_TMP}/${FILENAME}.${EXTENSION}" fi diff --git a/scripts/endOfNight.sh b/scripts/endOfNight.sh index e781ee80c..6e6267a3d 100755 --- a/scripts/endOfNight.sh +++ b/scripts/endOfNight.sh @@ -56,11 +56,11 @@ fi if [[ ${KEOGRAM} == "true" ]]; then echo -e "${ME}: ===== Generating Keogram" #shellcheck disable=SC2086 - "${ALLSKY_SCRIPTS}/generateForDay.sh" ${NICE_ARG} --silent -k "${DATE}" + "${ALLSKY_SCRIPTS}/generateForDay.sh" ${NICE_ARG} --silent --keogram "${DATE}" RET=$? echo -e "${ME}: ===== Keogram complete" if [[ ${UPLOAD_KEOGRAM} == "true" && ${RET} = 0 ]] ; then - "${ALLSKY_SCRIPTS}/generateForDay.sh" --upload -k "${DATE}" + "${ALLSKY_SCRIPTS}/generateForDay.sh" --upload --keogram "${DATE}" fi fi @@ -69,11 +69,11 @@ fi if [[ ${STARTRAILS} == "true" ]]; then echo -e "${ME}: ===== Generating Startrails" #shellcheck disable=SC2086 - "${ALLSKY_SCRIPTS}/generateForDay.sh" ${NICE_ARG} --silent -s "${DATE}" + "${ALLSKY_SCRIPTS}/generateForDay.sh" ${NICE_ARG} --silent --startrails "${DATE}" RET=$? echo -e "${ME}: ===== Startrails complete" if [[ ${UPLOAD_STARTRAILS} == "true" && ${RET} = 0 ]] ; then - "${ALLSKY_SCRIPTS}/generateForDay.sh" --upload -s "${DATE}" + "${ALLSKY_SCRIPTS}/generateForDay.sh" --upload --startrails "${DATE}" fi fi @@ -83,11 +83,11 @@ fi if [[ ${TIMELAPSE} == "true" ]]; then echo -e "${ME}: ===== Generating Timelapse" #shellcheck disable=SC2086 - "${ALLSKY_SCRIPTS}/generateForDay.sh" ${NICE_ARG} --silent -t "${DATE}" + "${ALLSKY_SCRIPTS}/generateForDay.sh" ${NICE_ARG} --silent --timelapse "${DATE}" RET=$? echo -e "${ME}: ===== Timelapse complete" if [[ ${UPLOAD_VIDEO} == "true" && ${RET} = 0 ]] ; then - "${ALLSKY_SCRIPTS}/generateForDay.sh" --upload -t "${DATE}" + "${ALLSKY_SCRIPTS}/generateForDay.sh" --upload --timelapse "${DATE}" fi fi diff --git a/scripts/flow-runner.py b/scripts/flow-runner.py index 3eef60032..c8070b813 100755 --- a/scripts/flow-runner.py +++ b/scripts/flow-runner.py @@ -7,6 +7,10 @@ import importlib from datetime import datetime, timedelta, date import signal +from collections import deque +import numpy +import shutil +import time ''' NOTE: `valid_module_paths` must be an array, and the order specified dictates the order of search for a named module. @@ -14,7 +18,6 @@ This permits the user to copy and modify a distributed module, or create an entirely new replacement for a distributed module, thus giving the user total control. ''' - def signalHandler(sig, frame): if sig == signal.SIGTERM or sig == signal.SIGINT: try: @@ -54,8 +57,25 @@ def signalHandler(sig, frame): if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument("-e", "--event", type=str, help="The event we are running modules for (defaults to postcapture).", default="postcapture", choices=["postcapture","daynight", "nightday", "periodic"]) + parser.add_argument("-f", "--flowtimerframes", type=int, help="Number of frames to capture for the flow timing averages.", default=10) + parser.add_argument("-c", "--cleartimings", action="store_true", help="Clear any flow average timing data.") shared.args = parser.parse_args() - + + shared.initDB() + + if shared.args.cleartimings: + if shared.dbHasKey("flowtimer"): + shared.dbDeleteKey("flowtimer") + + try: + flowTimingsFolder = os.environ["ALLSKY_FLOWTIMINGS"] + except KeyError: + flowTimingsFolder = os.path.join(shared.allskyTmp,"flowtimings") + + if os.path.exists(flowTimingsFolder): + shutil.rmtree(flowTimingsFolder) + sys.exit(0) + try: shared.allskyTmp = os.environ["ALLSKY_TMP"] except: @@ -69,7 +89,6 @@ def signalHandler(sig, frame): except: shared.log(0, "ERROR: no camera config file available in the environment", exitCode=1) - if (shared.args.event == "postcapture"): try: shared.LOGLEVEL = int(os.environ["ALLSKY_DEBUG_LEVEL"]) @@ -124,6 +143,7 @@ def signalHandler(sig, frame): shared.log(0, "ERROR: no allsky config directory available in the environment", exitCode=1) watchdog = False + moduleDebug = False timeout = 0 try: configFile = os.path.join(shared.args.allskyConfig, 'module-settings.json') @@ -131,10 +151,10 @@ def signalHandler(sig, frame): module_settings = json.load(module_Settings_file) watchdog = module_settings['watchdog'] timeout = module_settings['timeout'] + moduleDebug = module_settings['debugmode'] except: watchdog = False - - + shared.args.config = rawSettings shared.log(4, "INFO: Loading config {0}".format(shared.args.config)) try: @@ -158,8 +178,6 @@ def signalHandler(sig, frame): shared.log(0, "ERROR: Error parsing {0} {1}".format(moduleConfig, err), exitCode=1) except: shared.log(0, "ERROR: Failed to open {0}".format(moduleConfig), exitCode=1) - - shared.initDB() if (shared.args.event == "postcapture"): disableFile = os.path.join(shared.allskyTmp,"disable") @@ -173,13 +191,15 @@ def signalHandler(sig, frame): if hasattr(_temp, method): globals()[method] = getattr(_temp, method) result = globals()[method]() - shared.log(3, "INFO: Cleared module data for {0}".format(moduleName)) + shared.log(4, "INFO: Cleared module data for {0}".format(moduleName)) else: shared.log(3, "INFO: Attempting to clear module data for {0} but no function provided".format(moduleName)) os.remove(disableFile) results = {} + if moduleDebug: + flowStartTime = round(time.time() * 1000) for shared.step in shared.flow: if shared.flow[shared.step]["enabled"] and shared.flow[shared.step]["module"] not in globals(): try: @@ -192,7 +212,7 @@ def signalHandler(sig, frame): except Exception as e: shared.log(0, "ERROR: Failed to import module allsky_{0}.py in one of ( {1} ). Ignoring Module.".format(moduleName, e)) else: - shared.log(3, "INFO: Ignorning module {0} as its disabled".format(shared.flow[shared.step]["module"])) + shared.log(4, "INFO: Ignorning module {0} as its disabled".format(shared.flow[shared.step]["module"])) if shared.flow[shared.step]["enabled"] and method in globals(): startTime = datetime.now() @@ -201,7 +221,6 @@ def signalHandler(sig, frame): if 'arguments' in shared.flow[shared.step]['metadata']: arguments = shared.flow[shared.step]['metadata']['arguments'] - try: result = globals()[method](arguments, shared.args.event) except Exception as e: @@ -225,11 +244,11 @@ def signalHandler(sig, frame): shared.log(0, 'ERROR: Module {0} will be disabled, it took {1:.2f}s max allowed is {2}s'.format(shared.flow[shared.step]['module'], elapsedTime, timeout)) results[shared.step]["disable"] = True else: - shared.log(3, 'INFO: Module {0} ran ok in {1:.2f}s'.format(shared.flow[shared.step]['module'], elapsedTime)) + shared.log(4, 'INFO: Module {0} ran ok in {1:.2f}s'.format(shared.flow[shared.step]['module'], elapsedTime)) else: - shared.log(3, 'INFO: Module {0} ran ok in {1:.2f}s'.format(shared.flow[shared.step]['module'], elapsedTime)) + shared.log(4, 'INFO: Module {0} ran ok in {1:.2f}s'.format(shared.flow[shared.step]['module'], elapsedTime)) else: - shared.log(3, f'INFO: Ignoring watchdog for module {shared.step}') + shared.log(4, f'INFO: Ignoring watchdog for module {shared.step}') results[shared.step]["lastexecutiontime"] = str(elapsedTime) @@ -254,3 +273,50 @@ def signalHandler(sig, frame): json.dump(config, updatefile, indent=4) except json.JSONDecodeError as err: shared.log(0, "ERROR: Error parsing {0} {1}".format(moduleConfig, err), exitCode=1) + + if moduleDebug: + try: + flowTimingsFile = os.environ[f"ALLSKY_FLOWTIMINGS_{flowName.upper()}"] + + flowEndTime = round(time.time() * 1000) + flowElapsedTime = int(flowEndTime - flowStartTime) + queueData = [] + allQueueData = {} + if shared.dbHasKey("flowtimer"): + allQueueData = shared.dbGet("flowtimer") + if flowName in allQueueData: + queueData = allQueueData[flowName] + + queue = deque(queueData, maxlen = shared.args.flowtimerframes) + queue.append(flowElapsedTime) + + queueData = list(queue) + allQueueData[flowName] = queueData + shared.dbUpdate("flowtimer", allQueueData) + + try: + flowTimingsFolder = os.environ["ALLSKY_FLOWTIMINGS"] + except KeyError: + flowTimingsFolder = os.path.join(shared.allskyTmp,"flowtimings") + + shared.checkAndCreateDirectory(flowTimingsFolder) + if len(list(queue)) >= shared.args.flowtimerframes: + average = str(int(numpy.average(list(queue)))) + with open(flowTimingsFile, 'w') as f: + f.write(average) + else: + if shared.isFileWriteable(flowTimingsFile): + os.remove(flowTimingsFile) + except KeyError: + pass + + if not moduleDebug: + try: + flowTimingsFolder = os.environ["ALLSKY_FLOWTIMINGS"] + except KeyError: + flowTimingsFolder = os.path.join(shared.allskyTmp,"flowtimings") + if shared.dbHasKey("flowtimer"): + shared.dbDeleteKey("flowtimer") + + if os.path.exists(flowTimingsFolder): + shutil.rmtree(flowTimingsFolder) \ No newline at end of file diff --git a/scripts/functions.sh b/scripts/functions.sh index 44a87a710..798eb76ca 100644 --- a/scripts/functions.sh +++ b/scripts/functions.sh @@ -25,6 +25,15 @@ function doExit() COLOR="yellow" ;; esac + + OUTPUT_A_MSG="false" + if [[ -n ${WEBUI_MESSAGE} ]]; then + [[ ${TYPE} = "no-image" ]] && TYPE="success" + "${ALLSKY_SCRIPTS}/addMessage.sh" "${TYPE}" "${WEBUI_MESSAGE}" + echo "Stopping Allsky: ${WEBUI_MESSAGE}" + OUTPUT_A_MSG="true" + fi + if [[ ${EXITCODE} -ge ${EXIT_ERROR_STOP} ]]; then # With fatal EXIT_ERROR_STOP errors, we can't continue so display a notification image # even if the user has them turned off. @@ -36,18 +45,16 @@ function doExit() "${FILENAME:-"image"}" \ "${COLOR}" "" "85" "" "" \ "" "10" "${COLOR}" "${EXTENSION:-"jpg"}" "" "${CUSTOM_MESSAGE}" + echo "Stopping Allsky: ${CUSTOM_MESSAGE}" elif [[ ${TYPE} != "no-image" ]]; then + [[ ${OUTPUT_A_MSG} == "false" && ${TYPE} == "RebootNeeded" ]] && echo "Reboot needed" "${ALLSKY_SCRIPTS}/copy_notification_image.sh" --expires 0 "${TYPE}" 2>&1 fi - # Don't let the service restart us because we'll likely get the same error again. - echo " ***** AllSky Stopped *****" fi - if [[ -n ${WEBUI_MESSAGE} ]]; then - [[ ${TYPE} = "no-image" ]] && TYPE="success" - "${ALLSKY_SCRIPTS}/addMessage.sh" "${TYPE}" "${WEBUI_MESSAGE}" - fi + echo " ***** AllSky Stopped *****" + # Don't let the service restart us because we'll likely get the same error again. [[ ${EXITCODE} -ge ${EXIT_ERROR_STOP} ]] && sudo systemctl stop allsky # shellcheck disable=SC2086 @@ -70,6 +77,7 @@ function determineCommandToUse() # If it's not installed, or IS installed but doesn't work (the user may not have it configured), # use raspistill. + local RET=0 local CMD="libcamera-still" if command -v ${CMD} > /dev/null; then # Found the command - see if it works. @@ -88,7 +96,7 @@ function determineCommandToUse() CMD="raspistill" if ! command -v "${CMD}" > /dev/null; then - echo -e "${RED}*** ERROR: Can't determine what command to use for RPi camera.${NC}" + echo "Can't determine what command to use for RPi camera." >&2 if [[ ${USE_doExit} == "true" ]]; then doExit "${EXIT_ERROR_STOP}" "Error" "${PREFIX}\nRPi camera command\nnot found!." fi @@ -96,21 +104,17 @@ function determineCommandToUse() return 1 fi - # TODO: Should try and run raspistill command - doing that is more reliable since - # the output of vcgencmd changes depending on the OS and how the Pi is configured. - # Newer kernels/libcamera give: supported=1 detected=0, libcamera interfaces=1 - # but only if start_x=1 is in /boot/config.txt - vcgencmd get_camera | grep --silent "supported=1" ######### detected=1" + "${CMD}" --timeout 1 --nopreview > /dev/null 2>&1 RET=$? fi if [[ ${RET} -ne 0 ]]; then - echo -e "${RED}*** ERROR: RPi camera not found. Make sure it's enabled.${NC}" + echo "RPi camera not found. Make sure it's enabled." >&2 if [[ ${USE_doExit} == "true" ]]; then doExit "${EXIT_NO_CAMERA}" "Error" "${PREFIX}\nRPi camera\nnot found!\nMake sure it's enabled." fi - return 1 + return "${EXIT_NO_CAMERA}" fi echo "${CMD}" @@ -426,7 +430,7 @@ function get_variable() { local FILE="${2}" local LINE="" local SEARCH_STRING="^[ ]*${VARIABLE}=" - if ! LINE="$(grep -E "${SEARCH_STRING}" "${FILE}")" ; then + if ! LINE="$( /bin/grep -E "${SEARCH_STRING}" "${FILE}" )" ; then return 1 fi @@ -438,7 +442,359 @@ function get_variable() { # Simple way to get a setting that hides the details. function settings() { - j="$(jq -r "${1}" "${2:-${SETTINGS_FILE}}")" && echo "${j}" && return - echo "${ME2}: running as $(id --user --name), unable to get json value for '${1}';" >&2 - ls -l "${SETTINGS_FILE}" >&2 + local M="${ME:-settings}" + local FIELD="${1}" + # Arrays can't begin with period but everything else should. + if [[ ${FIELD:0:1} != "." && ${FIELD: -2:2} != "[]" ]]; then + echo "${M}: Field names must begin with period '.' (Field='${FIELD}')" >&2 + return 1 + fi + + local FILE="${2:-${SETTINGS_FILE}}" + if j="$( jq -r "${FIELD}" "${FILE}" )" ; then + echo "${j}" + return 0 + fi + + echo "${M}: Unable to get json value for '${FIELD}' in '${FILE}." >&2 + + return 2 +} + + +##### +# Return hard any link(s) to the specified file. +# The links must be in the same directory. +# On success return code 0 and the link(s). +# On failure, return code 1 and an error message. +NO_LINK_=3 +function get_links() +{ + local FILE="$1" + if [[ -z ${FILE} ]]; then + echo "get_links(): File not specified." + return 1 + fi + local DIRNAME="$( dirname "${FILE}" )" + + # shellcheck disable=SC2012 + local INODE="$( /bin/ls -l --inode "${FILE}" 2>/dev/null | cut -f1 -d' ' )" + if [[ -z ${INODE} ]]; then + echo "File '${FILE}' not found." + return 2 + fi + + # Don't include the specified FILE. + local LINKS="$( + if [[ ${DIRNAME} == "." ]]; then + x="./" + else + x="" + fi + find "${DIRNAME}" -inum "${INODE}" "!" -path "${x}${FILE}" | + if [[ -n ${x} ]]; then + sed -e "s;^${x};;" + else + cat + fi + )" + if [[ -z ${LINKS} ]]; then + echo "No links for '${FILE}'." + return "${NO_LINK_}" + fi + + echo "${LINKS}" + return 0 +} + + +##### +# Make sure the settings file is linked to the camera-specific file. +# Return 0 code and no message if successful, else 1 and return a message. +function check_settings_link() +{ + local FULL_FILE FILE DIRNAME SETTINGS_LINK RET MSG F E CORRECT_NAME + FULL_FILE="${1}" + if [[ -z ${FULL_FILE} ]]; then + echo "check_settings_link(): Settings file not specified." + return "${EXIT_ERROR_STOP}" + fi + if [[ -z ${CAMERA_TYPE} ]]; then + CAMERA_TYPE="$( settings .cameraType "${FULL_FILE}" )" + [[ $? -ne 0 || -z ${CAMERA_TYPE} ]] && return "${EXIT_ERROR_STOP}" + fi + if [[ -z ${CAMERA_MODEL} ]]; then + CAMERA_MODEL="$( settings .cameraModel "${FULL_FILE}" )" + [[ $? -ne 0 || -z ${CAMERA_TYPE} ]] && return "${EXIT_ERROR_STOP}" + fi + + DIRNAME="$( dirname "${FULL_FILE}" )" + FILE="$( basename "${FULL_FILE}" )" + F="${FILE%.*}" + E="${FILE##*.}" + CORRECT_NAME="${F}_${CAMERA_TYPE}_${CAMERA_MODEL}.${E}" + FULL_CORRECT_NAME="${DIRNAME}/${CORRECT_NAME}" + SETTINGS_LINK="$( get_links "${FULL_FILE}" )" + RET=$? + if [[ ${RET} -ne 0 ]]; then + MSG="The settings file '${FILE}' was not linked to '${CORRECT_NAME}'" + [[ ${RET} -ne "${NO_LINK_}" ]] && MSG="${MSG}\nERROR: ${SETTINGS_LINK}." + echo -e "${MSG}$( fix_settings_link "${FULL_FILE}" "${FULL_CORRECT_NAME}" )" + return 1 + else + # Make sure it's linked to the correct file. + if [[ ${SETTINGS_LINK} != "${FULL_CORRECT_NAME}" ]]; then + MSG="The settings file (${FULL_FILE}) was linked to:" + MSG="${MSG}\n ${SETTINGS_LINK}" + MSG="${MSG}\nbut should have been linked to:" + MSG="${MSG}\n ${FULL_CORRECT_NAME}" + echo -e "${MSG}$( fix_settings_link "${FULL_FILE}" "${FULL_CORRECT_NAME}" )" + return 1 + fi + fi + + return 0 +} + +function fix_settings_link() +{ + local SETTINGS="${1}" + local LINK="${2}" + + # Often the file to be linked to will exist, it just won't be linked. + # shellcheck disable=SC2012 + local NEWER="$( /bin/ls -t -1 "${SETTINGS}" "${LINK}" 2>/dev/null | head -1 )" + if [[ ${NEWER} == "${SETTINGS}" ]]; then + echo " but has been fixed." + rm -f "${LINK}" + ln "${SETTINGS}" "${LINK}" + else + # Typically the settings will will be newer than the camera-specific version. + echo " but has been fixed ('${LINK}' linked to '${SETTINGS}' - this is uncommon)." + rm -f "${SETTINGS}" + ln "${LINK}" "${SETTINGS}" + fi + + return 0 +} + +function update_json_file() # field, new value, file +{ + local M="${ME:-update_json_file}" + local FIELD="${1}" + if [[ ${FIELD:0:1} != "." ]]; then + echo "${M}: Field names must begin with period '.' (Field='${FIELD}')" >&2 + return 1 + fi + + local NEW_VALUE="${2}" + local FILE="${3:-${SETTINGS_FILE}}" + local TEMP="/tmp/$$" + # Have to use "cp" instead of "mv" to keep any hard link. + if jq "${FIELD} = \"${NEW_VALUE}\"" "${FILE}" > "${TEMP}" ; then + cp "${TEMP}" "${FILE}" + rm "${TEMP}" + return 0 + fi + + echo "${M}: Unable to update json value of '${FIELD}' to '${NEW_VALUE}' in '${FILE}'." >&2 + + return 2 +} + +#### +# Only allow one of the specified process at a time. +function one_instance() +{ + local SLEEP_TIME="5s" + local MAX_CHECKS=3 + local PID_FILE="" + local ABORTED_FILE="" + local ABORTED_FIELDS="" + local ABORTED_MSG1="" + local ABORTED_MSG2="" + local CAUSED_BY="" + local P="" + + OK="true" + local ERRORS="" + while [[ $# -gt 0 ]]; do + ARG="${1}" + case "${ARG}" in + --sleep) + SLEEP_TIME="${2}" + shift + ;; + --max-checks) + MAX_CHECKS=${2} + shift + ;; + --pid-file) + PID_FILE="${2}" + shift + ;; + --aborted-count-file) + ABORTED_FILE="${2}" + shift + ;; + --aborted-fields) + ABORTED_FIELDS="${2}" + shift + ;; + --aborted-msg1) + ABORTED_MSG1="${2}" + shift + ;; + --aborted-msg2) + ABORTED_MSG2="${2}" + shift + ;; + --caused-by) + CAUSED_BY="${2}" + shift + ;; + *) + ERRORS="${ERRORS}\nUnknown argument: '${ARG}'." + OK="false" + ;; + esac + shift + done + if [[ -z ${PID_FILE} ]]; then + ERRORS="${ERRORS}\nPID_FILE not specified." + OK="false" + fi + if [[ -z ${ABORTED_FILE} ]]; then + ERRORS="${ERRORS}\nABORTED_FILE not specified." + OK="false" + fi + if [[ -z ${ABORTED_FIELDS} ]]; then + ERRORS="${ERRORS}\nABORTED_FIELDS not specified." + OK="false" + fi + if [[ -z ${ABORTED_MSG1} ]]; then + ERRORS="${ERRORS}\nABORTED_MSG1 not specified." + OK="false" + fi + if [[ -z ${ABORTED_MSG2} ]]; then + ERRORS="${ERRORS}\nABORTED_MSG2 not specified." + OK="false" + fi + # CAUSED_BY isn't required + + if [[ ${OK} == "false" ]]; then + echo -e "${RED}${ME}: ERROR: ${ERRORS}.${NC}" >&2 + return 1 + fi + + + NUM_CHECKS=0 + while : ; do + [[ ! -f ${PID_FILE} ]] && break + + ((NUM_CHECKS++)) + + PID=$( < "${PID_FILE}" ) + # Check that the process is still running. + P="$( ps -fp "${PID}" )" + [[ $? -ne 0 ]] && break; # not running - why is the file still here? + + if [[ $NUM_CHECKS -eq ${MAX_CHECKS} ]]; then + echo -en "${YELLOW}" >&2 + echo -e "${ABORTED_MSG1}" >&2 + echo -n "Made ${NUM_CHECKS} attempts at waiting." >&2 + echo -n " If this happens often, check your settings." >&2 + echo -e "${NC}" >&2 + echo "${P}" >&2 + + # Keep track of aborts so user can be notified. + # If it's happening often let the user know. + [[ ! -d ${ALLSKY_ABORTS_DIR} ]] && mkdir "${ALLSKY_ABORTS_DIR}" + local AF="${ALLSKY_ABORTS_DIR}/${ABORTED_FILE}" + echo -e "$(date)\t${ABORTED_FIELDS}" >> "${AF}" + NUM=$( wc -l < "${AF}" ) + if [[ ${NUM} -eq 3 || ${NUM} -eq 10 ]]; then + MSG="${NUM} ${ABORTED_MSG2} have been aborted waiting for others to finish." + [[ -n ${CAUSED_BY} ]] && MSG="${MSG}\n${CAUSED_BY}" + if [[ ${NUM} -eq 3 ]]; then + SEVERITY="info" + else + SEVERITY="warning" + MSG="${MSG}\nOnce you have resolved the cause, reset the aborted counter:" + MSG="${MSG}\n    rm -f '${AF}'" + fi + "${ALLSKY_SCRIPTS}/addMessage.sh" "${SEVERITY}" "${MSG}" + fi + + return 2 + else + sleep "${SLEEP_TIME}" + fi + done + + echo $$ > "${PID_FILE}" || return 1 + + return 0 +} + + +##### +# Make a thumbnail image. +function make_thumbnail() +{ + local SEC="${1}" + local INPUT_FILE="${2}" + local THUMBNAIL="${3}" + ffmpeg -loglevel error -ss "00:00:${SEC}" -i "${INPUT_FILE}" \ + -filter:v scale="${THUMBNAIL_SIZE_X:-100}:-1" -frames:v 1 "${THUMBNAIL}" +} + + +##### +# Check if the user was supposed to reboot, and if so, if they did. +# Return 0 if a reboot is needed. +function reboot_needed() +{ + [[ ! -f ${ALLSKY_REBOOT_NEEDED} ]] && return 1 + + # The file exists so they were supposed to reboot. + BEFORE="$( < "${ALLSKY_REBOOT_NEEDED}" )" + NOW="$( uptime --since )" + if [[ ${BEFORE} == "${NOW}" ]]; then + return 0 + else + rm -f "${ALLSKY_REBOOT_NEEDED}" # different times so they rebooted + return 1 + fi +} + +#### +# Read json on stdin and output each field and value separated by a tab. +function convert_json_to_tabs() +{ + # Possible input formats, all with and without trailing "," and + # with or without leading spaces or tabs. + # "field" : "value" + # "field" : number + # "field": "value" + # "field": number + # "field":"value" + # "field":number + # Want to output two fields (field name and value), separated by tabs. + # First get rid of the brackets, + # then the optional leading spaces and tabs, + # then everything between the field and and its value, + # then ending " and/or comma. + + local JSON_FILE="${1}" + if [[ ! -f ${JSON_FILE} ]]; then + echo -e "${RED}convert_json_to_tabs(): ERROR: json file '${JSON_FILE}' not found.${NC}" >&2 + return 1 + fi + + sed -e '/^{/d' -e '/^}/d' \ + -e 's/^[\t ]*"//' \ + -e 's/"[\t :]*[ "]/\t/' \ + -e 's/",$//' -e 's/"$//' -e 's/,$//' \ + "${JSON_FILE}" } diff --git a/scripts/generateForDay.sh b/scripts/generateForDay.sh index a65d9d509..4aff29e0b 100755 --- a/scripts/generateForDay.sh +++ b/scripts/generateForDay.sh @@ -85,7 +85,8 @@ usage_and_exit() retcode=${1} echo [[ ${retcode} -ne 0 ]] && echo -en "${RED}" - echo "Usage: ${ME} [--help] [--silent] [--debug] [--nice n] [--upload] [--thumbnail-only] [-k] [-s] [-t] DATE" + echo "Usage: ${ME} [--help] [--silent] [--debug] [--nice n] [--upload] \\" + echo " [--thumbnail-only] [--keogram] [--startrails] [--timelapse] DATE" [[ ${retcode} -ne 0 ]] && echo -en "${NC}" echo " where:" echo " '--help' displays this message and exits." @@ -94,10 +95,10 @@ usage_and_exit() echo " '--upload' uploads previously-created files instead of creating them." echo " '--thumbnail-only' creates or uploads video thumbnails only." echo " 'DATE' is the day in '${ALLSKY_IMAGES}' to process." - echo " '-k' will ${MSG1} a keogram." - echo " '-s' will ${MSG1} a startrail." - echo " '-t' will ${MSG1} a timelapse." - echo " If you don't specify k, s, or t, all three will be ${MSG2}." + echo " '--keogram' will ${MSG1} a keogram." + echo " '--startrails' will ${MSG1} a startrail." + echo " '--timelapse' will ${MSG1} a timelapse." + echo " If you don't specify --keogram, --startrails, or --timelapse, all three will be ${MSG2}." # shellcheck disable=SC2086 exit ${retcode} } @@ -111,9 +112,9 @@ if [[ ${TYPE} == "UPLOAD" ]]; then fi DATE="${1}" -DATE_DIR="${ALLSKY_IMAGES}/${DATE}" -if [[ ! -d ${DATE_DIR} ]]; then - echo -e "${RED}${ME}: ERROR: '${DATE_DIR}' not found!${NC}" +OUTPUT_DIR="${ALLSKY_IMAGES}/${DATE}" +if [[ ! -d ${OUTPUT_DIR} ]]; then + echo -e "${RED}${ME}: ERROR: '${OUTPUT_DIR}' not found!${NC}" exit 2 fi @@ -122,7 +123,6 @@ if [[ ${GOT} -eq 0 ]]; then DO_STARTRAILS="true" DO_TIMELAPSE="true" fi -# echo -e "k=${DO_KEOGRAM}, s=${DO_STARTRAILS}, t=${DO_TIMELAPSE}\nDATE_DIR=${DATE_DIR}"; exit 0 if [[ ${TYPE} == "GENERATE" ]]; then generate() @@ -131,8 +131,9 @@ if [[ ${TYPE} == "GENERATE" ]]; then DIRECTORY="${2}" CMD="${3}" [[ ${SILENT} == "false" ]] && echo "===== Generating ${GENERATING_WHAT}" - [[ ${DIRECTORY} != "" ]] && mkdir -p "${DATE_DIR}/${DIRECTORY}" + [[ ${DIRECTORY} != "" ]] && mkdir -p "${OUTPUT_DIR}/${DIRECTORY}" + [[ -n ${DEBUG_ARG} ]] && echo "${ME}: Executing: ${CMD}" # shellcheck disable=SC2086 eval ${CMD} RET=$? @@ -161,7 +162,9 @@ else fi [[ ${SILENT} == "false" ]] && echo "===== Uploading '${UPLOAD_FILE}'" # shellcheck disable=SC2086 - "${ALLSKY_SCRIPTS}/upload.sh" ${UPLOAD_SILENT} ${DEBUG_ARG} "${UPLOAD_FILE}" "${DIRECTORY}" "${DESTINATION_NAME}" "${FILE_TYPE}" "${WEB_DIRECTORY}" + "${ALLSKY_SCRIPTS}/upload.sh" ${UPLOAD_SILENT} ${DEBUG_ARG} \ + "${UPLOAD_FILE}" "${DIRECTORY}" "${DESTINATION_NAME}" \ + "${FILE_TYPE}" "${WEB_DIRECTORY}" return $? else echo -en "${YELLOW}" @@ -192,51 +195,59 @@ fi if [[ ${DO_KEOGRAM} == "true" ]]; then KEOGRAM_FILE="keogram-${DATE}.${EXTENSION}" - UPLOAD_FILE="${DATE_DIR}/keogram/${KEOGRAM_FILE}" + UPLOAD_FILE="${OUTPUT_DIR}/keogram/${KEOGRAM_FILE}" if [[ ${TYPE} == "GENERATE" ]]; then if [[ -z "${NICE}" ]]; then N="" else N="--nice-level ${NICE}" fi - CMD="'${ALLSKY_BIN}/keogram' ${N} ${SIZE_FILTER} -d '${DATE_DIR}' -e ${EXTENSION} -o '${UPLOAD_FILE}' ${KEOGRAM_EXTRA_PARAMETERS}" + CMD="'${ALLSKY_BIN}/keogram' ${N} ${SIZE_FILTER} -d '${OUTPUT_DIR}' \ + -e ${EXTENSION} -o '${UPLOAD_FILE}' ${KEOGRAM_EXTRA_PARAMETERS}" generate "Keogram" "keogram" "${CMD}" else - upload "Keogram" "${UPLOAD_FILE}" "${KEOGRAM_DIR}" "${KEOGRAM_FILE}" "${KEOGRAM_DESTINATION_NAME}" "${WEB_KEOGRAM_DIR}" + upload "Keogram" "${UPLOAD_FILE}" "${KEOGRAM_DIR}" "${KEOGRAM_FILE}" \ + "${KEOGRAM_DESTINATION_NAME}" "${WEB_KEOGRAM_DIR}" fi [[ $? -ne 0 ]] && ((EXIT_CODE++)) fi if [[ ${DO_STARTRAILS} == "true" ]]; then STARTRAILS_FILE="startrails-${DATE}.${EXTENSION}" - UPLOAD_FILE="${DATE_DIR}/startrails/${STARTRAILS_FILE}" + UPLOAD_FILE="${OUTPUT_DIR}/startrails/${STARTRAILS_FILE}" if [[ ${TYPE} == "GENERATE" ]]; then if [[ -z "${NICE}" ]]; then N="" else N="--nice ${NICE}" fi - CMD="'${ALLSKY_BIN}/startrails' ${N} ${SIZE_FILTER} -d '${DATE_DIR}' -e ${EXTENSION} -b ${BRIGHTNESS_THRESHOLD} -o '${UPLOAD_FILE}' ${STARTRAILS_EXTRA_PARAMETERS}" + CMD="'${ALLSKY_BIN}/startrails' ${N} ${SIZE_FILTER} -d '${OUTPUT_DIR}' \ + -e ${EXTENSION} -b ${BRIGHTNESS_THRESHOLD} -o '${UPLOAD_FILE}' \ + ${STARTRAILS_EXTRA_PARAMETERS}" generate "Startrails, threshold=${BRIGHTNESS_THRESHOLD}" "startrails" "${CMD}" else - upload "Startrails" "${UPLOAD_FILE}" "${STARTRAILS_DIR}" "${STARTRAILS_FILE}" "${STARTRAILS_DESTINATION_NAME}" "${WEB_STARTRAILS_DIR}" + upload "Startrails" "${UPLOAD_FILE}" "${STARTRAILS_DIR}" "${STARTRAILS_FILE}" \ + "${STARTRAILS_DESTINATION_NAME}" "${WEB_STARTRAILS_DIR}" fi [[ $? -ne 0 ]] && ((EXIT_CODE++)) fi if [[ ${DO_TIMELAPSE} == "true" ]]; then VIDEOS_FILE="allsky-${DATE}.mp4" - UPLOAD_THUMBNAIL_NAME="allsky-${DATE}.jpg" - # Need a different name for the file on the Pi so it's not mistaken for a video file in the WebUI. + # Need a different name for the file so it's not mistaken for a regular image in the WebUI. THUMBNAIL_FILE="thumbnail-${DATE}.jpg" - UPLOAD_FILE="${DATE_DIR}/${VIDEOS_FILE}" - UPLOAD_THUMBNAIL="${DATE_DIR}/${THUMBNAIL_FILE}" + + UPLOAD_THUMBNAIL_NAME="allsky-${DATE}.jpg" + UPLOAD_THUMBNAIL="${OUTPUT_DIR}/${THUMBNAIL_FILE}" + UPLOAD_FILE="${OUTPUT_DIR}/${VIDEOS_FILE}" + if [[ ${TYPE} == "GENERATE" ]]; then if [[ ${THUMBNAIL_ONLY} == "true" ]]; then if [[ -f ${UPLOAD_FILE} ]]; then RET=0 else - echo -e "${RED}${ME}: ERROR: video file '${UPLOAD_FILE}' not found!\nCannot create thumbnail.${NC}" + echo -e "${RED}${ME}: ERROR: video file '${UPLOAD_FILE}' not found!" + echo -e "Cannot create thumbnail.${NC}" RET=1 fi else @@ -245,23 +256,19 @@ if [[ ${DO_TIMELAPSE} == "true" ]]; then else N="nice -n ${NICE}" fi - CMD="${N} '${ALLSKY_SCRIPTS}/timelapse.sh' ${DATE}" + CMD="${N} '${ALLSKY_SCRIPTS}/timelapse.sh' --output '${UPLOAD_FILE}' ${DATE}" generate "Timelapse" "" "${CMD}" # it creates the necessary directory RET=$? fi if [[ ${RET} -eq 0 && ${TIMELAPSE_UPLOAD_THUMBNAIL} == "true" ]]; then rm -f "${UPLOAD_THUMBNAIL}" - function make_thumbnail() - { - local SEC="${1}" - ffmpeg -loglevel error -ss "00:00:${SEC}" -i "${UPLOAD_FILE}" \ - -filter:v scale="${THUMBNAIL_SIZE_X}:-1" -frames:v 1 "${UPLOAD_THUMBNAIL}" - } # Want the thumbnail to be near the start of the video, but not the first frame # since that can be a lousy frame. # If the video is less than 5 seconds, make_thumbnail won't work, so try again. - make_thumbnail 5 - [[ ! -f ${UPLOAD_THUMBNAIL} ]] && make_thumbnail 0 + make_thumbnail "05" "${UPLOAD_FILE}" "${UPLOAD_THUMBNAIL}" + if [[ ! -f ${UPLOAD_THUMBNAIL} ]]; then + make_thumbnail "00" "${UPLOAD_FILE}" "${UPLOAD_THUMBNAIL}" + fi if [[ ! -f ${UPLOAD_THUMBNAIL} ]]; then echo -e "${RED}${ME}: ERROR: video thumbnail not created!${NC}" fi @@ -270,7 +277,8 @@ if [[ ${DO_TIMELAPSE} == "true" ]]; then if [[ ${THUMBNAIL_ONLY} == "true" ]]; then RET=0 else - upload "Timelapse" "${UPLOAD_FILE}" "${VIDEOS_DIR}" "${VIDEOS_FILE}" "${VIDEOS_DESTINATION_NAME}" "${WEB_VIDEOS_DIR}" + upload "Timelapse" "${UPLOAD_FILE}" "${VIDEOS_DIR}" "${VIDEOS_FILE}" \ + "${VIDEOS_DESTINATION_NAME}" "${WEB_VIDEOS_DIR}" RET=$? fi if [[ ${RET} -eq 0 && ${TIMELAPSE_UPLOAD_THUMBNAIL} == "true" && -f ${UPLOAD_THUMBNAIL} ]]; then @@ -279,7 +287,8 @@ if [[ ${DO_TIMELAPSE} == "true" ]]; then else W="" fi - upload "TimelapseThumbnail" "${UPLOAD_THUMBNAIL}" "${VIDEOS_DIR}/thumbnails" "${UPLOAD_THUMBNAIL_NAME}" "" "${W}" + upload "TimelapseThumbnail" "${UPLOAD_THUMBNAIL}" "${VIDEOS_DIR}/thumbnails" \ + "${UPLOAD_THUMBNAIL_NAME}" "" "${W}" fi fi [[ ${RET} -ne 0 ]] && ((EXIT_CODE++)) @@ -288,9 +297,9 @@ fi if [[ ${TYPE} == "GENERATE" && ${SILENT} == "false" && ${EXIT_CODE} -eq 0 ]]; then ARGS="${THUMBNAIL_ONLY_ARG}" - [[ ${DO_KEOGRAM} == "true" ]] && ARGS="${ARGS} -k" - [[ ${DO_STARTRAILS} == "true" ]] && ARGS="${ARGS} -s" - [[ ${DO_TIMELAPSE} == "true" ]] && ARGS="${ARGS} -t" + [[ ${DO_KEOGRAM} == "true" ]] && ARGS="${ARGS} --keogram" + [[ ${DO_STARTRAILS} == "true" ]] && ARGS="${ARGS} --startrails" + [[ ${DO_TIMELAPSE} == "true" ]] && ARGS="${ARGS} --timelapse" echo -e "\n================" echo "If you want to upload the file(s) you just created," echo -e "\texecute '${ME} --upload ${ARGS} ${DATE}'" diff --git a/scripts/generate_notification_images.sh b/scripts/generate_notification_images.sh index 783fa3e44..ef156cb36 100755 --- a/scripts/generate_notification_images.sh +++ b/scripts/generate_notification_images.sh @@ -26,7 +26,7 @@ function usage_and_exit() ( [[ ${RET} -ne 0 ]] && echo -en "${RED}" echo -e "\nUsage: ${ME} [--help] [--directory dir] [--size XxY]" - echo -e "\t[type TextColor Font FontSize StrokeColor StrokeWidth BgColor BorderWidth BorderColor Extensions ImageSize 'Message']\n" + echo -e " [type TextColor Font FontSize StrokeColor StrokeWidth BgColor BorderWidth BorderColor Extensions ImageSize 'Message']\n" [[ ${RET} -ne 0 ]] && echo -en "${NC}" echo "When run with no arguments, all notification types are created with extensions: ${ALL_EXTS/ /, }." echo "Arguments:" @@ -67,10 +67,13 @@ while [[ $# -gt 0 ]]; do IMAGE_SIZE="${X}x${Y}" shift ;; - *) - display_msg error "Unknown argument: '${ARG}'." + -*) + echo "${RED}${ME}: ERROR: Unknown argument: '${ARG}'${NC}." >&2 OK="false" ;; + *) + break + ;; esac shift done diff --git a/scripts/installUpgradeFunctions.sh b/scripts/installUpgradeFunctions.sh index fc1076b8d..47945d9ee 100644 --- a/scripts/installUpgradeFunctions.sh +++ b/scripts/installUpgradeFunctions.sh @@ -43,12 +43,13 @@ function get_Git_version() { } +# For the version, if the last character of the argument passed is a "/", +# then append the file name from ${ALLSKY_VERSION_FILE}. +# We do this to limit the number of places that know the actual name of the file, +# in case we ever change it. + ##### # Get the version from a local file, if it exists. -# For the version and branch, if the last character of the argument passed is a "/", -# then append the file name fro ${ALLSKY_VERSION_FILE} or ${ALLSKY_BRANCH_FILE}. -# We do this to limit the number of places that know the actual name of the files, -# in case we every change their names. function get_version() { local F="${1}" if [[ -z ${F} ]]; then @@ -57,26 +58,21 @@ function get_version() { [[ ${F:1,-1} == "/" ]] && F="${F}$(basename "${ALLSKY_VERSION_FILE}")" fi if [[ -f ${F} ]]; then - local VALUE="$( < "${F}" )" + # Sometimes the branch file will have both "master" and "dev" on two lines. + local VALUE="$( head -1 "${F}" )" echo -n "${VALUE}" | tr -d '\n\r' fi } ##### -# Get the branch from a local file, if it exists. +# Get the branch using git. function get_branch() { - local F="${1}" - if [[ -z ${F} ]]; then - F="${ALLSKY_BRANCH_FILE}" # default - else - [[ ${F:1,-1} == "/" ]] && F="${F}$(basename "${ALLSKY_BRANCH_FILE}")" - fi - - # Branch file is same format as Version file. - echo -n "$(get_version "${F}")" + local H="${1:-${ALLSKY_HOME}}" + echo "$( cd "${H}" || exit; git rev-parse --abbrev-ref HEAD )" } + ##### # Display a message of various types in appropriate colors. # Used primarily in installation scripts. @@ -120,14 +116,15 @@ function display_msg() STARS=false elif [[ ${LOG_TYPE} == "info" ]]; then - LOGMSG="${MESSAGE}" + LOGMSG="* ${MESSAGE}" MSG="${YELLOW}${LOGMSG}${NC}" STARS=false elif [[ ${LOG_TYPE} == "debug" ]]; then # Indent so they align with text above LOGMSG=" DEBUG: ${MESSAGE}" - MSG="${YELLOW}${LOGMSG}${NC}" + #shellcheck disable=SC2154 + MSG="${cDEBUG}${LOGMSG}${NC}" STARS=false else @@ -152,8 +149,33 @@ function display_msg() # Log messages to a file if it was specified. # ${DISPLAY_MSG_LOG} be set if ${LOG_IT} is true, but just in case, check. + if [[ ${LOG_IT} == "true" && -n ${DISPLAY_MSG_LOG} ]]; then - echo -e "${LOGMSG}${MESSAGE2}" >> "${DISPLAY_MSG_LOG}" + # Strip out all color escape sequences before adding to log file. + # This requires escaping the "\" (which appear at the beginning of every variable) + # and "[" in the variables. + + echo "${LOGMSG}${MESSAGE2}" | + ( + if [[ -n ${GREEN} ]]; then + # In case a variable isn't define, set it to a string that won't be found + local Y="${YELLOW:-abcxyz}" + local R="${RED:-abcxyz}" + local D="${cDEBUG:-abcxyz}" + local N="${NC:-abcxyz}" + + # I couldn't figure out how to replace "\n" with a new line in sed. + O="$( sed \ + -e "s/\\${GREEN/\[/\\[}//g" \ + -e "s/\\${Y/\[/\\[}//g" \ + -e "s/\\${R/\[/\\[}//g" \ + -e "s/\\${D/\[/\\[}//g" \ + -e "s/\\${N/\[/\\[}//g" )" + echo -e "${O}" # handles the newlines + else + cat + fi + ) >> "${DISPLAY_MSG_LOG}" fi } diff --git a/scripts/makeChanges.sh b/scripts/makeChanges.sh index 8d47699f9..8770f5291 100755 --- a/scripts/makeChanges.sh +++ b/scripts/makeChanges.sh @@ -82,6 +82,11 @@ if [[ ${OPTIONS_FILE_ONLY} == "false" ]]; then [[ $(($# % 4)) -ne 0 ]] && usage_and_exit 2 fi +if [[ ${ON_TTY} -eq 0 ]]; then # called from WebUI. + ERROR_PREFIX="" +else + ERROR_PREFIX="${ME}: " +fi # This output may go to a web page, so use "w" colors. # shell check doesn't realize there were set in variables.sh @@ -160,6 +165,7 @@ while [[ $# -gt 0 ]]; do CAMERA_NUMBER=" -cameraNumber ${NEW_CAMERA_NUMBER}" # Set NEW_VALUE to the current Camera Type NEW_VALUE="$( settings .cameraType )" + MSG="Re-creating files for cameraType ${NEW_VALUE}, cameraNumber ${NEW_CAMERA_NUMBER}" if [[ ${ON_TTY} -eq 0 ]]; then # called from WebUI. echo -e "" @@ -168,6 +174,13 @@ while [[ $# -gt 0 ]]; do fi fi + if [[ ! -e "${ALLSKY_BIN}/capture_${NEW_VALUE}" ]]; then + MSG="Unknown Camera Type: '${NEW_VALUE}'." + echo -e "${wERROR}${ERROR_PREFIX}ERROR: ${MSG}${wNC}" + # shellcheck disable=SC2086 + exit ${EXIT_NO_CAMERA} + fi + # This requires Allsky to be stopped so we don't # try to call the capture program while it's already running. sudo systemctl stop allsky 2> /dev/null @@ -177,8 +190,24 @@ while [[ $# -gt 0 ]]; do # If we can't set the new camera type, it's a major problem so exit right away. # NOTE: when we're changing cameraType we're not changing anything else. - CC_FILE_OLD="${CC_FILE}-OLD" + # The software for RPi cameras needs to know what command is being used to + # capture the images. + # determineCommandToUse either retuns the command with exit code 0, + # or an error message with non-zero exit code. + if [[ ${NEW_VALUE} == "RPi" ]]; then + C="$( determineCommandToUse "false" "" )" + RET=$? + if [[ ${RET} -ne 0 ]] ; then + echo -e "${wERROR}${ERROR_PREFIX}ERROR: ${C}.${wNC}" + # shellcheck disable=SC2086 + exit ${RET} + fi + C=" -cmd ${C}" + else + C="" + fi + CC_FILE_OLD="${CC_FILE}-OLD" if [[ -f ${CC_FILE} ]]; then # Save the current file just in case creating a new one fails. # It's a link so copy it to a temp name, then remove the old name. @@ -189,18 +218,6 @@ while [[ $# -gt 0 ]]; do # Create the camera capabilities file for the new camera type. # Use Debug Level 3 to give the user more info on error. - # The software for RPi cameras needs to know what command is being used to - # capture the images. - if [[ ${NEW_VALUE} == "RPi" ]]; then - if ! C="$(determineCommandToUse "false" "" )" ; then - echo -e "${wERROR}${ME}: ERROR: unable to determine command to use, RET=${RET}, C=${C}.${wNC}." - # shellcheck disable=SC2086 - exit ${RET} - fi - C=" -cmd ${C}" - else - C="" - fi if [[ ${DEBUG} == "true" ]]; then echo -e "${wDEBUG}Calling capture_${NEW_VALUE}${C}${CAMERA_NUMBER} -cc_file '${CC_FILE}'${wNC}" fi @@ -275,8 +292,13 @@ while [[ $# -gt 0 ]]; do fi # createAllskyOptions.php will use the cc file and the options template file - # to create an OPTIONS_FILE for this camera type/model. - # These variables don't include a directory since the directory is specified with "--dir" below. + # to create an OPTIONS_FILE and SETTINGS_FILE for this camera type/model. + # If there is an existing camera-specific settings file for the new + # camera type/model then createAllskyOptions.php will use it and link it + # to SETTINGS_FILE. + # If there is no existing camera-specific file, i.e., this camera is new + # to Allsky, it will create a default settings file using the generic + # values from the prior settings file if it exists. if [[ ${DEBUG} == "true" ]]; then # shellcheck disable=SC2086 @@ -285,7 +307,8 @@ while [[ $# -gt 0 ]]; do ${FORCE} ${DEBUG_ARG} \ "\n\t--cc_file ${CC_FILE}" \ "\n\t--options_file ${OPTIONS_FILE}" \ - "\n\t--settings_file ${SETTINGS_FILE}${wNC}" + "\n\t--settings_file ${SETTINGS_FILE}" \ + "${wNC}" fi # shellcheck disable=SC2086 R="$("${ALLSKY_WEBUI}/includes/createAllskyOptions.php" \ @@ -310,11 +333,11 @@ while [[ $# -gt 0 ]]; do OK="true" if [[ ! -f ${OPTIONS_FILE} ]]; then - echo -e "${wERROR}${ME}: ERROR Options file ${OPTIONS_FILE} not created.${wNC}" + echo -e "${wERROR}${ERROR_PREFIX}ERROR Options file ${OPTIONS_FILE} not created.${wNC}" OK="false" fi if [[ ! -f ${SETTINGS_FILE} && ${OPTIONS_FILE_ONLY} == "false" ]]; then - echo -e "${wERROR}${ME}: ERROR Settings file ${SETTINGS_FILE} not created.${wNC}" + echo -e "${wERROR}${ERROR_PREFIX}ERROR Settings file ${SETTINGS_FILE} not created.${wNC}" OK="false" fi [[ ${OK} == "false" ]] && exit 2 @@ -505,7 +528,7 @@ if [[ ${#WEBSITE_CONFIG[@]} -gt 0 ]]; then "RemoteWebsite" R=$? if [[ ${R} -ne 0 ]]; then - echo -e "${RED}${ME}: Unable to upload '${FILE_TO_UPLOAD}'.${NC}" + echo -e "${RED}${ERROR_PREFIX}Unable to upload '${FILE_TO_UPLOAD}'.${NC}" fi fi fi diff --git a/scripts/modules/allsky_overlay.py b/scripts/modules/allsky_overlay.py index 1909c7b4f..977ccce0f 100644 --- a/scripts/modules/allsky_overlay.py +++ b/scripts/modules/allsky_overlay.py @@ -97,7 +97,10 @@ def __init__(self, formaterrortext): self._overlayConfigFile = os.path.join(os.environ['ALLSKY_OVERLAY'], 'config', self._OVERLAYCONFIGFILE) fieldsFile = os.path.join(os.environ['ALLSKY_OVERLAY'], 'config', self._OVERLAYFIELDSFILE) userFieldsFile = os.path.join(os.environ['ALLSKY_OVERLAY'], 'config', self._OVERLAYUSERFIELDSFILE) - self._OVERLAYTMP = os.path.join(tempfile.gettempdir(), 'overlay') + + tmpFolder = os.path.join(os.environ['ALLSKY_OVERLAY'], 'tmp') + self._createTempDir(tmpFolder) + self._OVERLAYTMP = os.path.join(tmpFolder, 'overlay') self._createTempDir(self._OVERLAYTMP) self._OVERLAYTLEFOLDER = os.path.join(self._OVERLAYTMP , 'tle') self._createTempDir(self._OVERLAYTLEFOLDER) @@ -109,13 +112,13 @@ def __init__(self, formaterrortext): self._fields = self._systemfields + self._userfields - s.log(4, "INFO: Config file set to {}".format(self._overlayConfigFile)) + s.log(4, "INFO: Config file set to '{}'.".format(self._overlayConfigFile)) self._enableSkyfield = True try: load = Loader(self._OVERLAYTMP, verbose=False) self._eph = load('de421.bsp') except Exception as err: - s.log(1, "ERROR: Unable to download de421.bsp. {}".format(err)) + s.log(0, "ERROR: Unable to download de421.bsp: {}".format(err)) self._enableSkyfield = False self._setDateandTime() self._observerLat = s.getSetting('latitude') @@ -280,7 +283,7 @@ def _readData(self, dataFilename, defaultExpiry): os.environ[name] = str(value) self._saveExtraDataField(name, fileModifiedTime, expires, x, y, fill, font, fontsize, image, rotate, scale, opacity, stroke, strokewidth) else: - s.log(0, 'ERROR: Data File {} is not accessible - IGNORING'.format(dataFilename)) + s.log(0, 'ERROR: Data File {} is not accessible - IGNORING.'.format(dataFilename)) result = False else: if s.isFileReadable(dataFilename): @@ -294,7 +297,7 @@ def _readData(self, dataFilename, defaultExpiry): os.environ[name] = str(value) self._saveExtraDataField(name, fileModifiedTime, defaultExpiry) else: - s.log(0, "ERROR: Data File {} is not accessible - IGNORING".format(dataFilename)) + s.log(0, "ERROR: Data File {} is not accessible - IGNORING.".format(dataFilename)) result = False return result @@ -318,7 +321,7 @@ def _saveExtraDataField(self, name, fieldDate, expires, x=None, y=None, fill=Non 'strokewidth': strokewidth } self._extraData[name] = _extraField - s.log(4,"INFO: Added extra data field {0}, expiry {1} seconds".format(name, expires)) + s.log(4,"INFO: Added extra data field {0}, expiry {1} seconds.".format(name, expires)) def _loadConfigFile(self): result = True @@ -327,10 +330,10 @@ def _loadConfigFile(self): self._overlayConfig = json.load(file) if len(self._overlayConfig["fields"]) == 0 and len(self._overlayConfig["images"]) == 0: - s.log(1, "WARNING: Config file ({}) is empty.".format(self._overlayConfigFile)) + s.log(1, "WARNING: Config file '{}' is empty.".format(self._overlayConfigFile)) result = True else: - s.log(0, "ERROR: Config File not accessible {}".format(self._overlayConfigFile)) + s.log(0, "ERROR: Config File '{}' not accessible.".format(self._overlayConfigFile)) result = False return result @@ -363,9 +366,9 @@ def _timer(self, text, showIntermediate = True, showMessage=True): elapsedTime = datetime.now() - self._startTime if showIntermediate: - s.log(4, "INFO: {0} took {1} Seconds. Elapsed Time {2} Seconds.".format(text, lastText, elapsedTime.total_seconds())) + s.log(4, "INFO: {0} took {1} seconds. Elapsed time {2} seconds.".format(text, lastText, elapsedTime.total_seconds())) else: - s.log(4, "INFO: {0} Elapsed Time {1} Seconds.".format(text, elapsedTime.total_seconds())) + s.log(4, "INFO: {0} Elapsed time {1} seconds.".format(text, elapsedTime.total_seconds())) def _getFont(self, font, fontSize): @@ -381,8 +384,7 @@ def _getFont(self, font, fontSize): 'Comic Sans MS': {'fontpath': '/usr/share/fonts/truetype/msttcorefonts/comic.ttf'}, } - s.log(4, "INFO: Loading Font {0} Size {1} pixels".format(font, fontSize), True) - + preMsg = "Loading '{0}' font, size {1} pixels".format(font, fontSize) fontPath = None if font in self._overlayConfig['fonts']: fontData = self._overlayConfig['fonts'][font] @@ -394,7 +396,7 @@ def _getFont(self, font, fontSize): if font in systemFontMap: fontPath = systemFontMap[font]['fontpath'] else: - s.log(1, ', ERROR: System Font could not be found in map') + s.log(0, "ERROR: System font '{}' not found in internal map.".format(font)) if fontPath is not None: if fontSize is None: @@ -407,15 +409,15 @@ def _getFont(self, font, fontSize): if fontKey in self._fonts: font = self._fonts[fontKey] - s.log(4, ', Font Found In Cache') + s.log(4, F'{preMsg} - found in cache.') else : try: fontSize = s.int(fontSize) self._fonts[fontKey] = ImageFont.truetype(fontPath, fontSize) font = self._fonts[fontKey] - s.log(4, ', Font loaded from disk') + s.log(4, F'{preMsg} - loaded from disk.') except OSError as err: - s.log(1, ', ERROR: Font could not be loaded from disk ({})'.format(fontPath)) + s.log(0, "ERROR: Font '{}' could not be loaded from disk.".format(fontPath)) font = None else: font = None @@ -620,7 +622,10 @@ def _convertColour(self, name, value): try: r,g,b = ImageColor.getcolor(value, "RGB") except: - s.log(0, "ERROR: The colour '{0}' for field '{1}' Is NOT valid - Defaulting to white".format(value, name)) + if value == "": + s.log(0, "ERROR: The colour for field '{0}' is empty - Defaulting to white.".format(name)) + else: + s.log(0, "ERROR: The colour '{0}' for field '{1}' is NOT valid - Defaulting to white.".format(value, name)) r = 255 g = 255 b = 255 @@ -669,7 +674,7 @@ def _doBoolFormat(self, field, value, format): elif format == '%1': v = '1' else: - s.log(0, f"ERROR: Cannot use format '{format}' on Bool variables like {field}.") + s.log(0, f"ERROR: Cannot use format '{format}' on Bool variables like {field} (value={value}).") v = self._formaterrortext elif format == '%yes': @@ -681,7 +686,7 @@ def _doBoolFormat(self, field, value, format): elif format == '%1': v = '0' else: - s.log(0, f"ERROR: Cannot use format '{format}' on Bool variables like {field}.") + s.log(0, f"ERROR: Cannot use format '{format}' on Bool variables like {field} (value={value}).") v = self._formaterrortext return v @@ -793,7 +798,7 @@ def _getValue(self, placeHolder, variableType, format=None, empty=''): if format is None: value = time.strftime('%H:%M:%S', timeStamp) else: - # TODO: Check for bad format? + # TODO: Check for bad format? value = time.strftime(format, timeStamp) else: pass @@ -811,10 +816,10 @@ def _getValue(self, placeHolder, variableType, format=None, empty=''): try: value = format.format(convertValue) except Exception as err: - s.log(0, f"ERROR: Cannot use format '{f}' on Number variables like {rawFieldName}.") + s.log(0, f"ERROR: Cannot use format '{f}' on Number variables like {rawFieldName} (value={value}).") value = self._formaterrortext except ValueError as err: - s.log(0, f"ERROR: Cannot use format '{f}' on Number variables like {rawFieldName}.") + s.log(0, f"ERROR: Cannot use format '{f}' on Number variables like {rawFieldName} (value={value}).") if variableType == 'Bool': if format is None or format == '': @@ -827,7 +832,7 @@ def _getValue(self, placeHolder, variableType, format=None, empty=''): value = empty elif variableType is None or variableType == '': value = '???' - s.log(0, f"ERROR: {rawFieldName} has no variable type; check 'fields.json'") + s.log(0, f"ERROR: {rawFieldName} has no variable type; check 'fields.json'. Using '{value}' instead.") return value, x, y, fill, font, fontsize, rotate, scale, opacity, stroke, strokewidth @@ -849,8 +854,8 @@ def _addImages(self): def _doAddImage(self, imageData): imageName = imageData["image"] if imageName != 'missing': - imageX = imageData["x"] - imageY = imageData["y"] + imageX = int(imageData["x"]) + imageY = int(imageData["y"]) image = None imagePath = os.path.join(os.environ['ALLSKY_OVERLAY'], "images", imageName) @@ -866,7 +871,7 @@ def _doAddImage(self, imageData): if "rotate" in imageData: if imageData["rotate"] is not None: - image = self._rotate_image(image, imageData["rotate"]) + image = self._rotate_image(image, int(imageData["rotate"])) height = image.shape[0] width = image.shape[1] @@ -878,9 +883,9 @@ def _doAddImage(self, imageData): s.log(4, "INFO: Adding image field {}".format(imageName)) else: - s.log(0, "ERROR: Cannot locate image {}".format(imageName)) + s.log(0, "ERROR: Cannot locate image {}.".format(imageName)) else: - s.log(0, "ERROR: Image not set so ignoring") + s.log(0, "ERROR: Image not set so ignoring.") def _overlay_transparent(self, imageName, background, overlay, x, y, imageData): @@ -926,7 +931,7 @@ def _overlay_transparent(self, imageName, background, overlay, x, y, imageData): background[y:y+h, x:x+w] = (1.0 - mask) * background[y:y+h, x:x+w] + mask * overlay_image else: - s.log(0, "ERROR: Adding image {}. Its outside the bounds of the main image".format(imageName)) + s.log(0, "ERROR: Image '{}' is outside the bounds of the main image.".format(imageName)) return background def _rotate_image(self, image, angle): @@ -979,9 +984,9 @@ def _initialiseMoon(self): os.environ['AS_MOON_SYMBOL'] = self._moonPhaseSymbol s.log(4, 'INFO: Adding Moon Symbol {}'.format(self._moonPhaseSymbol)) else: - s.log(4,'INFO: Moon enabled but cannot use') + s.log(4,'INFO: Moon enabled but cannot use due to prior error.') else: - s.log(4,'INFO: Moon not enabled') + s.log(4,'INFO: Moon not enabled.') return True def _fileCreatedToday(self, fileName): @@ -1136,9 +1141,9 @@ def _initialiseSunOld(self): os.environ[key] = value s.log(4, 'INFO: Adding {0}:{1}'.format(key,value)) else: - s.log(4,'INFO: Sun enabled but cannot use') + s.log(4,'INFO: Sun enabled but cannot use due to prior error.') else: - s.log(4,'INFO: Sun not enabled') + s.log(4,'INFO: Sun not enabled.') return True @@ -1226,12 +1231,12 @@ def _initSatellites(self): else: os.environ['AS_' + noradId + 'VISIBLE'] = 'No' except LookupError: - s.log(4,'ERROR: Norad ID ' + noradId + ' Not found') + s.log(0, 'ERROR: Norad ID ' + noradId + ' Not found.') else: - s.log(4,'INFO: Satellites enabled but cannot use') + s.log(4, 'INFO: Satellites enabled but cannot use due to prior error.') else: - s.log(4,'INFO: Satellites not enabled') + s.log(4, 'INFO: Satellites not enabled.') return True @@ -1274,9 +1279,9 @@ def _initPlanets(self): else: os.environ['AS_' + planetId.replace(' BARYCENTER','') + 'VISIBLE'] = 'No' else: - s.log(4,'INFO: Planets enabled but unable to use') + s.log(4, 'INFO: Planets enabled but unable to use due to prior error.') else: - s.log(4,'INFO: Planets not enabled') + s.log(4, 'INFO: Planets not enabled.') return True @@ -1327,5 +1332,5 @@ def overlay(params, event): else: result = "External Overlay Disabled" - s.log(4,"INFO: {0}".format(result)) + s.log(4,"INFO: {0}.".format(result)) return result diff --git a/scripts/modules/allsky_shared.py b/scripts/modules/allsky_shared.py index b743ed5dc..ae159f624 100644 --- a/scripts/modules/allsky_shared.py +++ b/scripts/modules/allsky_shared.py @@ -351,6 +351,12 @@ def dbUpdate(key, value): DBDATA[key] = value writeDB() +def dbDeleteKey(key): + global DBDATA + if dbHasKey(key): + del DBDATA[key] + writeDB() + def dbHasKey(key): global DBDATA return (key in DBDATA) diff --git a/scripts/removeBadImages.sh b/scripts/removeBadImages.sh index 5ef645331..40ef0d43a 100755 --- a/scripts/removeBadImages.sh +++ b/scripts/removeBadImages.sh @@ -1,28 +1,42 @@ #!/bin/bash +# Remove "bad" images, which includes: +# Empty files +# Corrupted files (i.e., not an image) +# Too dark (as specified by user) +# Too light (as specified by user) + +# If a file is specified, only look at that file, +# otherwise look at all the files in the specified directory. +# If only 1 file, return it's MEAN. + +# The MEAN is the only thing that should go to stdout. + ME="$(basename "${BASH_ARGV0}")" #shellcheck disable=SC2086 source-path=. -source "${ALLSKY_HOME}/variables.sh" || exit ${ALLSKY_ERROR_STOP} +source "${ALLSKY_HOME}/variables.sh" || exit ${ALLSKY_ERROR_STOP} #shellcheck disable=SC2086 source-path=scripts source "${ALLSKY_SCRIPTS}/functions.sh" || exit ${ALLSKY_ERROR_STOP} -#shellcheck disable=SC2086,SC1091 # file doesn't exist in GitHub -source "${ALLSKY_CONFIG}/config.sh" || exit ${ALLSKY_ERROR_STOP} +#shellcheck disable=SC2086,SC1091 # file doesn't exist in GitHub +source "${ALLSKY_CONFIG}/config.sh" || exit ${ALLSKY_ERROR_STOP} usage() { retcode="${1}" - echo - echo "Remove images with corrupt data which might mess up startrails and keograms." - [ "${retcode}" -ne 0 ] && echo -en "${RED}" - echo -n "Usage: ${ME} [--help] [--debug] directory [file]" - [ "${retcode}" -ne 0 ] && echo -e "${NC}" - echo - echo "You must enter the arguments in the above order." + ( + echo + echo "Remove images with corrupt data which might mess up startrails and keograms." + [ "${retcode}" -ne 0 ] && echo -en "${RED}" + echo -n "Usage: ${ME} [--help] [--debug] directory [file]" + [ "${retcode}" -ne 0 ] && echo -e "${NC}" + echo + echo "You must enter the arguments in the above order." # TODO: use getopts to allow any order - echo "Turning on debug will indicate bad images but will not remove them." - echo "If 'file' is specified, only that file in 'directory' will be checked," - echo "otherwise all files in 'directory' will be checked." + echo "Turning on debug will indicate bad images but will not remove them." + echo "If 'file' is specified, only that file in 'directory' will be checked," + echo "otherwise all files in 'directory' will be checked." + ) >&2 # shellcheck disable=SC2086 exit ${retcode} } @@ -48,15 +62,15 @@ else ME="${ME}:" fi -# If it's not a full pathname, assume it's in $ALLSKY_IMAGES which is what many other scripts do. +# If it's not a full pathname, assume it's in $ALLSKY_IMAGES. [[ ${DATE:0:1} != "/" ]] && DATE="${ALLSKY_IMAGES}/${DATE}" if [[ ! -d ${DATE} ]]; then - echo -e "${RED}${ME} '${DATE}' is not a directory${NC}" + echo -e "${RED}${ME} '${DATE}' is not a directory${NC}" >&2 exit 2 fi if [[ ${FILE} != "" && ! -f ${DATE}/${FILE} ]]; then - echo -e "${RED}${ME} '${FILE}' not found in '${DATE}'${NC}" + echo -e "${RED}${ME} '${FILE}' not found in '${DATE}'${NC}" >&2 exit 2 fi @@ -99,14 +113,14 @@ else # looking at a directory fi num_bad=0 -# If the low threshold is 0 it's disabled. -# If the high one is 0 or 100 (nothing can be brighter than 100) it's disabled. -if [[ ${REMOVE_BAD_IMAGES_THRESHOLD_HIGH} -gt 100 || ${REMOVE_BAD_IMAGES_THRESHOLD_HIGH} -eq 0 ]]; then - HIGH=0 -else - HIGH=${REMOVE_BAD_IMAGES_THRESHOLD_HIGH} -fi + +# If the LOW threshold is 0 it's disabled. +# If the HIGH threshold is 0 or 100 (nothing can be brighter than 100) it's disabled. +# Convert possibly 0.0 and 100.0 to 0 and 100 so we can check using bash. +HIGH=${REMOVE_BAD_IMAGES_THRESHOLD_HIGH} +[[ $( echo "${HIGH} == 0 || ${HIGH} > 100" | bc ) -eq 1 ]] && HIGH=0 LOW=${REMOVE_BAD_IMAGES_THRESHOLD_LOW} +[[ $( echo "${LOW} <= 0" | bc ) -eq 1 ]] && LOW=0 # If we're processing a whole directory assume it's done in the background so "nice" it. # If we're only processing one file we want it done quickly. @@ -122,33 +136,40 @@ for f in ${IMAGE_FILES} ; do if [[ ! -s ${f} ]]; then BAD="'${f}' (zero length)" else - # MEAN is a number between 0.0 and 1.0. if ! MEAN=$(${NICE} convert "${f}" -colorspace Gray -format "%[fx:image.mean]" info: 2>&1) ; then # Do NOT set BAD since this isn't necessarily a problem with the file. - echo -e "${RED}***${ME}: ERROR: 'convert ${f}' failed; leaving file.${NC}" - echo "Message=${MEAN}" + echo -e "${RED}***${ME}: ERROR: 'convert ${f}' failed; leaving file.${NC}" >&2 + echo -e "Message=${MEAN}" >&2 elif [[ -z ${MEAN} ]]; then # Do NOT set BAD since this isn't necessarily a problem with the file. - echo -e "${RED}***${ME}: ERROR: 'convert ${f}' returned nothing; leaving file.${NC}" + echo -e "${RED}***${ME}: ERROR: 'convert ${f}' returned nothing; leaving file.${NC}" >&2 elif echo "${MEAN}" | grep -E -q "${ERROR_WORDS}"; then # At least one error word was found in the output. # Get rid of unnecessary error text, and only look at first line of error message. BAD="'${f}' (corrupt file: $(echo "${MEAN}" | sed -e 's;convert-im6.q16: ;;' -e 's; @ error.*;;' -e 's; @ warning.*;;' -e q))" else - # MEAN is a number between 0.0 and 1.0 but it may have format: + # If only one file, output its mean. + [[ ${FILE} != "" ]] && echo "${MEAN}" + + # MEAN is a number between 0.0 and 1.0, but it may have format: # 6.90319e-06 # which "bc" doesn't accept. - # Since the shell doesn't do floating point math and we want the user to be able to specify - # up to two digits precision, multiple the MEAN and the user's numbers by 100. + # LOW and HIGH are from 0 to 100 so first multiple the MEAN by 100 + # to match the LOW and HIGH. + MEAN=$( echo "${MEAN}" | awk '{ printf("%0.2f", $1 * 100); }' ) + + # Since the shell doesn't do floating point math and we want the user to + # be able to specify up to two digits precision, + # multiple everything by 100 and convert to integer. # Awk handles the "e-" format. - MEAN100=$( echo "${MEAN}" | awk '{ printf("%d", $1 * 100); }' ) - HIGH100=$( echo "${HIGH}" | awk '{ printf("%d", $1 * 100); }' ) - LOW100=$( echo "${LOW}" | awk '{ printf("%d", $1 * 100); }' ) + MEAN_CHECK=$( echo "${MEAN}" | awk '{ printf("%d", $1 * 100); }' ) + HIGH_CHECK=$( echo "${HIGH}" | awk '{ printf("%d", $1 * 100); }' ) + LOW_CHECK=$( echo "${LOW}" | awk '{ printf("%d", $1 * 100); }' ) MSG="" if [[ ${HIGH} != "0" ]]; then # Use the HIGH check - if [[ ${MEAN100} -gt "${HIGH100}" ]]; then + if [[ $(echo "${MEAN_CHECK} > ${HIGH_CHECK}" | bc ) -eq 1 ]]; then BAD="'${f}' (above threshold: MEAN=${MEAN}, threshold = ${HIGH})" elif [[ ${DEBUG} == "true" ]]; then MSG="===== OK: ${f}, MEAN=${MEAN}, HIGH=${HIGH}, LOW=${LOW}" @@ -157,7 +178,7 @@ for f in ${IMAGE_FILES} ; do # An image can't be both HIGH and LOW so if it was HIGH don't check for LOW. if [[ ${BAD} == "" && ${LOW} != "0" ]]; then - if [[ ${MEAN100} -lt "${LOW100}" ]]; then + if [[ $(echo "${MEAN_CHECK} < ${LOW_CHECK}" | bc ) -eq 1 ]]; then BAD="'${f}' (below threshold: MEAN=${MEAN}, threshold = ${LOW})" elif [[ ${DEBUG} == "true" && ${MSG} == "" ]]; then MSG="===== OK: ${f}, MEAN=${MEAN}, HIGH=${HIGH}, LOW=${LOW}" @@ -165,7 +186,7 @@ for f in ${IMAGE_FILES} ; do fi if [[ ${DEBUG} == "true" && ${BAD} == "" && -n ${MSG} ]]; then - echo "${MSG}" + echo "${MSG}" >&2 fi fi fi @@ -177,22 +198,47 @@ for f in ${IMAGE_FILES} ; do echo "${r} ${BAD}" >> "${OUTPUT}" fi [[ ${DEBUG} == "false" ]] && rm -f "${f}" "thumbnails/${f}" - num_bad=$((num_bad + 1)) + ((num_bad++)) fi done if [[ $num_bad -eq 0 ]]; then # If only one file, "no news is good news". if [[ -z ${FILE} ]]; then - echo -e "\n${ME} ${GREEN}No bad files found.${NC}" + echo -e "\n${ME} ${GREEN}No bad files found.${NC}" >&2 rm -f "${OUTPUT}" + else + rm -f "${ALLSKY_BAD_IMAGE_COUNT}" fi else if [[ -z ${FILE} ]]; then - echo "${ME} ${num_bad} bad file(s) found and ${r}. See ${OUTPUT}." + echo "${ME} ${num_bad} bad file(s) found and ${r}. See ${OUTPUT}." >&2 # Do NOT remove ${OUTPUT} in case the user wants to look at it. else # only 1 file so show it - echo "${ME} File is bad: ${OUTPUT}" + echo "${ME} File is bad: ${OUTPUT}" >&2 + echo -e "${OUTPUT}" >> "${ALLSKY_BAD_IMAGE_COUNT}" + BAD_COUNT="$( wc -l < "${ALLSKY_BAD_IMAGE_COUNT}" )" + # TODO: make BAD_LIMIT a WebUI setting. + BAD_LIMIT=3 +# echo "xxxxxxxxxx ${BAD_COUNT} bad consecutive images" >&2 + if [[ $((BAD_COUNT % BAD_LIMIT)) -eq 0 ]]; then + MSG="Multiple bad consecutive bad images." + MSG="${MSG}\nCheck 'REMOVE_BAD_IMAGES_THRESHOLD_LOW' and 'REMOVE_BAD_IMAGES_THRESHOLD_HIGH' in config.sh" + "${ALLSKY_SCRIPTS}/addMessage.sh" "warning" "${MSG}" >&2 + fi + if [[ ${BAD_COUNT} -ge "${BAD_LIMIT}" ]]; then + # Split the long file name so it fits in the message. + DIR="$( dirname "${ALLSKY_BAD_IMAGE_COUNT}" )" + FILE="$( basename "${ALLSKY_BAD_IMAGE_COUNT}" )" + + "${ALLSKY_SCRIPTS}/generate_notification_images.sh" \ + --directory "${ALLSKY_TMP}" \ + "${FILENAME}" "yellow" "" "85" "" "" \ + "" "5" "yellow" "${EXTENSION}" "" \ + "WARNING:\n${BAD_COUNT} consecutive\nbad images. See:\n${DIR}/\n ${FILE}" >&2 + + fi + fi fi diff --git a/scripts/saveImage.sh b/scripts/saveImage.sh index f8b044e9d..b2959e2c9 100755 --- a/scripts/saveImage.sh +++ b/scripts/saveImage.sh @@ -4,7 +4,7 @@ ME="$(basename "${BASH_ARGV0}")" -[[ ${ALLSKY_DEBUG_LEVEL} -ge 4 ]] && echo "${ME} $*" +[[ ${ALLSKY_DEBUG_LEVEL} -ge 3 ]] && echo "${ME} $*" #shellcheck disable=SC2086 source-path=. source "${ALLSKY_HOME}/variables.sh" || exit ${ALLSKY_ERROR_STOP} @@ -17,12 +17,13 @@ usage_and_exit() { retcode=${1} [[ ${retcode} -ne 0 ]] && echo -ne "${RED}" - echo -n "Usage: ${ME} DAY|NIGHT full_path_to_filename [variable=value [...]]" + echo -n "Usage: ${ME} DAY|NIGHT full_path_to_image [variable=value [...]]" [[ ${retcode} -ne 0 ]] && echo -e "${NC}" # shellcheck disable=SC2086 exit ${retcode} } [[ $# -lt 2 ]] && usage_and_exit 1 + # Export so other scripts can use it. export DAY_OR_NIGHT="${1}" [[ ${DAY_OR_NIGHT} != "DAY" && ${DAY_OR_NIGHT} != "NIGHT" ]] && usage_and_exit 1 @@ -45,6 +46,27 @@ if [[ ! -s ${CURRENT_IMAGE} ]] ; then exit 2 fi +# Make sure only one save happens at once. +# Multiple concurrent saves (which can happen if the delay is short or post-processing +# is long) causes read and write errors. +PID_FILE="${ALLSKY_TMP}/saveImage-pid.txt" +ABORTED_MSG1="Another saveImage is in progress so the new one was aborted." +ABORTED_FIELDS="${CURRENT_IMAGE}" +ABORTED_MSG2="uploads" +# TODO: check delay settings and average times for module processing +# and tailor the message. +CAUSED_BY="This could be caused by very long module processing time or extremely short delays between images." +# Don't sleep too long or check too many times since processing an image should take at most +# a few seconds +if ! one_instance --pid-file "${PID_FILE}" --sleep "3s" --max-checks 3 \ + --aborted-count-file "${ALLSKY_ABORTEDSAVEIMAGE}" --aborted-fields "${ABORTED_FIELDS}" \ + --aborted-msg1 "${ABORTED_MSG1}" --aborted-msg2 "${ABORTED_MSG2}" \ + --caused-by "${CAUSED_BY}" ; then + rm -f "${CURRENT_IMAGE}" + exit 1 +fi + + # The image may be in a memory filesystem, so do all the processing there and # leave the image used by the website(s) in that directory. IMAGE_NAME=$(basename "${CURRENT_IMAGE}") # just the file name @@ -53,9 +75,15 @@ WORKING_DIR=$(dirname "${CURRENT_IMAGE}") # the directory the image is currently # Optional full check for bad images. if [[ ${REMOVE_BAD_IMAGES} == "true" ]]; then # If the return code is 99, the file was bad and deleted so don't continue. - "${ALLSKY_SCRIPTS}/removeBadImages.sh" "${WORKING_DIR}" "${IMAGE_NAME}" + AS_MEAN2="$( "${ALLSKY_SCRIPTS}/removeBadImages.sh" "${WORKING_DIR}" "${IMAGE_NAME}" )" # removeBadImages.sh displayed error message and deleted the file. - [[ $? -eq 99 ]] && exit 99 + if [[ $? -eq 99 ]]; then + exit 99 + elif [[ -n ${AS_MEAN2} ]]; then + export AS_MEAN2 + fi +else + AS_MEAN2="" fi # If we didn't execute removeBadImages.sh do a quick sanity check on the image. @@ -90,9 +118,28 @@ done # Export other variables so user can use them in overlays export AS_CAMERA_TYPE="${CAMERA_TYPE}" export AS_CAMERA_MODEL="${CAMERA_MODEL}" +if [[ -n ${AS_MEAN2} ]]; then + export AS_MEAN_NORMALIZED="$( echo "${AS_MEAN2} * 255" | bc )" # xxxx for testing +fi + + +# If ${AS_TEMPERATURE_C} is set, use it as the sensor temperature, +# otherwise use the temperature in ${TEMPERATURE_FILE}. +# TODO: Currently nothing creates the TEMPERATURE_FILE. Eventually RPi cameras will. +if [[ -z ${AS_TEMPERATURE_C} ]]; then + TEMPERATURE_FILE="${ALLSKY_TMP}/temperature.txt" + if [[ -s ${TEMPERATURE_FILE} ]]; then # -s so we don't use an empty file + AS_TEMPERATURE_C=$( < "${TEMPERATURE_FILE}" ) + fi +fi + +# If taking dark frames, save the dark frame then exit. +if [[ $(settings ".takeDarkFrames") -eq 1 ]]; then + #shellcheck source-path=scripts + source "${ALLSKY_SCRIPTS}/darkCapture.sh" + exit 0 +fi -#shellcheck source-path=scripts -source "${ALLSKY_SCRIPTS}/darkCapture.sh" # does not return if in darkframe mode # TODO: Dark subtract long-exposure images, even if during daytime. # TODO: Need a config variable to specify the threshold to dark subtract. # TODO: Possibly also for stretching below. @@ -216,6 +263,11 @@ fi "${ALLSKY_SCRIPTS}/flow-runner.py" +# The majority of the post-processing time for an image is in flow-runner.py. +# Since only one mini-timelapse can run at once and that code is embeded in this code +# in several places, remove our PID lock now. +rm -f "${PID_FILE}" + SAVED_FILE="${CURRENT_IMAGE}" # The name of the file saved from the camera. WEBSITE_FILE="${WORKING_DIR}/${FULL_FILENAME}" # The name of the file the websites look for @@ -267,7 +319,7 @@ if [[ ${SAVE_IMAGE} == "true" ]]; then else if ! grep --silent "${FINAL_FILE}" "${MINI_TIMELAPSE_FILES}" ; then echo "${FINAL_FILE}" >> "${MINI_TIMELAPSE_FILES}" - elif [[ ${ALLSKY_DEBUG_LEVEL} -ge 2 ]]; then + elif [[ ${ALLSKY_DEBUG_LEVEL} -ge 1 ]]; then # This shouldn't happen... echo -e "${YELLOW}${ME} WARNING: '${FINAL_FILE}' already in set.${NC}" >&2 fi @@ -298,25 +350,32 @@ if [[ ${SAVE_IMAGE} == "true" ]]; then else D="" fi + O="${ALLSKY_TMP}/mini-timelapse.mp4" # shellcheck disable=SC2086 - "${ALLSKY_SCRIPTS}"/timelapse.sh ${D} --mini "${MINI_TIMELAPSE_FILES}" "${DATE_NAME}" + "${ALLSKY_SCRIPTS}"/timelapse.sh ${D} --lock --output "${O}" \ + --mini --images "${MINI_TIMELAPSE_FILES}" RET=$? - [[ ${RET} -ne 0 ]] && TIMELAPSE_MINI_UPLOAD_VIDEO="false" # failed so don't try to upload + if [[ ${RET} -ne 0 ]]; then + # failed so don't try to upload + TIMELAPSE_MINI_UPLOAD_VIDEO="false" + fi if [[ ${ALLSKY_DEBUG_LEVEL} -ge 2 ]]; then if [[ ${RET} -eq 0 ]]; then - echo "${ME}: mini-timelapse created" + echo "${ME}: mini-timelapse created (last image: ${IMAGE_NAME})" else - echo "${ME}: mini-timelapse creation returned with RET=${RET}" + echo "${ME}: mini-timelapse creation returned with RET=${RET} (last image: ${IMAGE_NAME})" fi fi - # Remove the oldest files, but not if we only created this mini-timelapse because of a force - if [[ ${MOD} -ne 0 || ${TIMELAPSE_MINI_FORCE_CREATION} == "false" ]]; then + # Remove the oldest files, but not if we only created + # this mini-timelapse because of a force. + if [[ ${RET} -eq 0 && (${MOD} -ne 0 || ${TIMELAPSE_MINI_FORCE_CREATION} == "false") ]]; then KEEP=$((TIMELAPSE_MINI_IMAGES - TIMELAPSE_MINI_FREQUENCY)) x="$(tail -${KEEP} "${MINI_TIMELAPSE_FILES}")" echo -e "${x}" > "${MINI_TIMELAPSE_FILES}" - if [[ ${ALLSKY_DEBUG_LEVEL} -ge 2 ]]; then - echo -e "${YELLOW}${ME} Replaced ${TIMELAPSE_MINI_FREQUENCY} oldest file(s) and added current image.${NC}" >&2 + if [[ ${ALLSKY_DEBUG_LEVEL} -ge 4 ]]; then + echo -en "${YELLOW}${ME}: Replaced ${TIMELAPSE_MINI_FREQUENCY} oldest" + echo -e " file(s) and added current image.${NC}" >&2 fi fi else @@ -415,8 +474,7 @@ if [[ ${TIMELAPSE_MINI_UPLOAD_VIDEO} == "true" && ${SAVE_IMAGE} == "true" && ${R UPLOAD_THUMBNAIL="${ALLSKY_TMP}/${UPLOAD_THUMBNAIL_NAME}" # Create the thumbnail for the mini timelapse, then upload it. rm -f "${UPLOAD_THUMBNAIL}" - ffmpeg -loglevel error -i "${FILE_TO_UPLOAD}" \ - -filter:v scale="${THUMBNAIL_SIZE_X}:-1" -frames:v 1 "${UPLOAD_THUMBNAIL}" + make_thumbnail "00" "${FILE_TO_UPLOAD}" "${UPLOAD_THUMBNAIL}" if [[ ! -f ${UPLOAD_THUMBNAIL} ]]; then echo "${ME}Mini timelapse thumbnail not created!" else diff --git a/scripts/timelapse.sh b/scripts/timelapse.sh index 3308523d5..36e7d6d04 100755 --- a/scripts/timelapse.sh +++ b/scripts/timelapse.sh @@ -12,10 +12,13 @@ source "${ALLSKY_SCRIPTS}/functions.sh" || exit ${ALLSKY_ERROR_STOP} source "${ALLSKY_CONFIG}/config.sh" || exit ${ALLSKY_ERROR_STOP} +ENTERED="$*" DEBUG=0 HELP="false" -DO_MINI="false" -MINI_FILE="" +IS_MINI="false" +LOCK="false" +IMAGES_FILE="" +OUTPUT_FILE="" while [[ $# -gt 0 ]]; do case "${1}" in -h | --help) @@ -24,11 +27,20 @@ while [[ $# -gt 0 ]]; do -d | --debug) ((DEBUG++)) ;; - -m | --mini) - DO_MINI="true" - MINI_FILE="${2}" + -l | --lock) + LOCK="true" + ;; + -o | --output) + OUTPUT_FILE="${2}" shift ;; + -i | --images) + IMAGES_FILE="${2}" + shift + ;; + -m | --mini) + IS_MINI="true" + ;; -*) echo -e "${RED}${ME}: Unknown argument '${1}' ignoring.${NC}" >&2 HELP="true" @@ -43,166 +55,174 @@ done usage_and_exit() { RET=$1 - XD="/path/to/nonstandard/location/of/allsky_images" + XD="/some_nonstandard_path" TODAY="$(date +%Y%m%d)" [[ ${RET} -ne 0 ]] && echo -en "${RED}" - echo -n "Usage: ${ME} [--debug] [--help] [--mini mini_file] []" + echo -n "Usage: ${ME} [--debug] [--help] [--lock] [--output file] [--mini] {--images file | }" echo -e "${NC}" echo " example: ${ME} ${TODAY}" - echo " or: ${ME} ${TODAY} ${XD}" + echo " or: ${ME} --output '${XD}' ${TODAY}" echo echo -en "${YELLOW}" - echo " must be of the form YYYYMMDD." echo - echo " defaults to '\${ALLSKY_IMAGES}' but may be overridden to use a" - echo "nonstandard location such as a usb stick or a network drive (eg. for regenerating timelapses)." - echo "In that case must exist inside ," - echo "eg. '${XD}/${TODAY}'." + echo "You entered: ${ME} ${ENTERED}" + echo + echo "The list of images is determined in one of two ways:" + echo "1. Looking in '' for files with an extension of '${EXTENSION}'." + echo " If is NOT a full path name it is assumed to be in '${ALLSKY_IMAGES}'," + echo " which allows using images on a USB stick, for example." + echo " The timelapse file is stored in and is called 'allsky-.mp4'," + echo " where is the basename of ." echo - echo "Produces a movie in //allsky-.mp4" - echo "eg. ${ALLSKY_IMAGES}/${TODAY}/allsky-${TODAY}.mp4" - echo "or ${XD}/${TODAY}/allsky-${TODAY}.mp4" + echo "2. Specifying '--images file' uses the images listed in 'file'; is not used." + echo " The timelapse file is stored in the same directory as the first image." echo - echo "[--mini mini_file] creates a mini-timelapse using the images listed in 'mini-file'." + echo "'--lock' ensures only one instance of ${ME} runs at a time." + echo "'--output file' overrides the default storage location and file name." + echo "'--mini' uses the MINI_TIMELAPSE settings and the timelapse file is" + echo " called 'mini-timelapse.mp4' if '--output' isn't used." echo -en "${NC}" # shellcheck disable=SC2086 exit ${RET} } -[[ $# -eq 0 || $# -gt 2 ]] && usage_and_exit 1 +if [[ -n ${IMAGES_FILE} ]]; then + # If IMAGES_FILE is specified there should be no other arguments. + [[ $# -ne 0 ]] && usage_and_exit 1 +elif [[ $# -eq 0 || $# -gt 1 ]]; then + usage_and_exit 2 +fi [[ ${HELP} == "true" ]] && usage_and_exit 0 -# Allow timelapses of pictures not in the standard $ALLSKY_IMAGES directory. -# If $2 is passed, it's the top folder, otherwise use the one in $ALLSKY_IMAGES. -DATE="${1}" -if [[ -z ${2} ]]; then - DATE_DIR="${ALLSKY_IMAGES}/${DATE}" # Need full pathname for links +OUTPUT_DIR="" +if [[ -n ${IMAGES_FILE} ]]; then + if [[ ! -s ${IMAGES_FILE} ]]; then + echo -e "${RED}*** ${ME} ERROR: '${IMAGES_FILE}' does not exist or is empty!${NC}" + exit 3 + fi + INPUT_DIR="" # Not used else - DATE_DIR="${2}/${DATE}" -fi -if [[ ! -d ${DATE_DIR} ]]; then - echo -e "${RED}*** ${ME} ERROR: '${DATE_DIR}' does not exist!${NC}" - exit 2 -fi + INPUT_DIR="${1}" -if [[ ${DO_MINI} == "false" ]]; then - OUTPUT_FILE="${DATE_DIR}/allsky-${DATE}.mp4" -else - # In MINI mode, only allow one process at a time. - OUTPUT_FILE="${ALLSKY_TMP}/mini-timelapse.mp4" - PID_FILE="${ALLSKY_TMP}/timelapse-mini-pid.txt" - NUM_CHECKS=0 - MAX_CHECKS=2 - SLEEP_TIME="15s" + # If not a full pathname, ${DIRNAME} will be "." so look in ${ALLSKY_IMAGES}. + DIRNAME="$( dirname "${INPUT_DIR}" )" + if [[ ${DIRNAME} == "." ]]; then + INPUT_DIR="${ALLSKY_IMAGES}/${INPUT_DIR}" # Need full pathname for links + fi + OUTPUT_DIR="${INPUT_DIR}" # default location - while : ; do - [[ ! -f ${PID_FILE} ]] && break + if [[ ! -d ${INPUT_DIR} ]]; then + echo -e "${RED}*** ${ME} ERROR: '${INPUT_DIR}' does not exist!${NC}" + exit 4 + fi +fi - PID=$( < "${PID_FILE}" ) - # shellcheck disable=SC2009 - if ! ps -fp "${PID}" | grep -E --silent "${ME}.*--mini" ; then - break # Not sure why the PID file existed if the process didn't exist. - fi +if [[ ${LOCK} == "true" ]]; then + PID_FILE="${ALLSKY_TMP}/timelapse-pid.txt" + ABORTED_MSG1="Another timelapse creation is in progress so this one was aborted." + ABORTED_FIELDS="$( basename "${OUTPUT_FILE}" )" + ABORTED_MSG2="timelapse creations" + if [[ ${IS_MINI} == "true" ]]; then + CAUSED_BY="This could be caused by unreasonable TIMELAPSE_MINI_IMAGES and TIMELAPSE_MINI_FREQUENCY settings." + else + CAUSED_BY="Unknown cause - see /var/log/allsky.log." + fi + if ! one_instance --pid-file "${PID_FILE}" \ + --aborted-count-file "${ALLSKY_ABORTEDTIMELAPSE}" \ + --aborted-fields "${ABORTED_FIELDS}" \ + --aborted-msg1 "${ABORTED_MSG1}" --aborted-msg2 "${ABORTED_MSG2}" \ + --caused-by "${CAUSED_BY}" ; then + exit 5 + fi + SEQUENCE_DIR="${ALLSKY_TMP}/sequence-lock-timelapse" +else + SEQUENCE_DIR="${ALLSKY_TMP}/sequence-timelapse" + PID_FILE="" +fi - if [[ $NUM_CHECKS -eq ${MAX_CHECKS} ]]; then - echo -en "${YELLOW}" >&2 - echo -n "${ME}: WARNING: Another mini timelapse creation is in progress" >&2 - echo " so this one was aborted." >&2 - echo "If this happens often, check your network and delay settings" >&2 - echo -n "as well as your TIMELAPSE_MINI_IMAGES and TIMELAPSE_MINI_FREQUENCY settings." >&2 - echo -e "${NC}" >&2 - ps -fp "${PID}" >&2 +if [[ -z ${OUTPUT_FILE} ]]; then + if [[ ${IS_MINI} == "true" ]]; then + OUTPUT_DIR="${ALLSKY_TMP}" + OUTPUT_FILE="${OUTPUT_DIR}/mini-timelapse.mp4" + else + if [[ -n ${IMAGES_FILE} ]]; then + # Use the directory the images are in. Only look at the first one. + I="$( head -1 "${IMAGES_FILE}" )" + OUTPUT_DIR="$( dirname "${I}" )" - # Keep track of aborts so user can be notified. - # If it's happening often let the user know. - echo -e "$(date)\t${OUTPUT_FILE}" >> "${ALLSKY_ABORTEDTIMELAPSE}" - NUM=$( wc -l < "${ALLSKY_ABORTEDTIMELAPSE}" ) - if [[ ${NUM} -eq 3 || ${NUM} -eq 10 ]]; then - MSG="${NUM} mini timelapse creations have been aborted waiting for others to finish." - MSG="${MSG}\nThis could be caused by unreasonable" - MSG="${MSG} TIMELAPSE_MINI_IMAGES and TIMELAPSE_MINI_FREQUENCY settings." - if [[ ${NUM} -eq 3 ]]; then - SEVERITY="info" - else - SEVERITY="warning" - MSG="${MSG}\nOnce you have resolved the cause, reset the aborted counter:" - MSG="${MSG}\n    rm -f '${ALLSKY_ABORTEDTIMELAPSE}'" - fi - "${ALLSKY_SCRIPTS}/addMessage.sh" "${SEVERITY}" "${MSG}" + # In case the filename doesn't include a path, put in a default location. + if [[ ${OUTPUT_DIR} == "." ]]; then + OUTPUT_DIR="${ALLSKY_TMP}" + echo -en "${ME}: ${YELLOW}" + echo "Can't determine where to put timelapse file so putting in '${OUTPUT_DIR}'." + echo -e "${NC}" fi - - exit 2 - else - sleep "${SLEEP_TIME}" fi - ((NUM_CHECKS++)) - done - echo $$ > "${PID_FILE}" || exit 1 -fi - -# To save on writes to SD card for people who have $ALLSKY_TMP as a memory filesystem, -# put the sequence files there. -SEQUENCE_DIR="${ALLSKY_TMP}/sequence-${DATE}-$$" -if [[ -d ${SEQUENCE_DIR} ]]; then - NSEQ=$(find "${SEQUENCE_DIR}/*" 2>/dev/null | wc -l) # left over from last time -else - NSEQ=0 + # Use the basename of the directory. + B="$( basename "${OUTPUT_DIR}" )" + OUTPUT_FILE="${OUTPUT_DIR}/allsky-${B}.mp4" + fi fi TMP="${ALLSKY_TMP}/timelapseTMP.txt" -[[ ${DO_MINI} == "false" ]] && : > "${TMP}" # Only create when NOT doing mini-timelapses +[[ ${IS_MINI} == "false" ]] && : > "${TMP}" # Only create when NOT doing mini-timelapses -if [[ ${KEEP_SEQUENCE} == "false" || ${NSEQ} -lt 100 ]]; then +if [[ ${KEEP_SEQUENCE} == "false" ]]; then rm -fr "${SEQUENCE_DIR}" mkdir -p "${SEQUENCE_DIR}" + NUM_IMAGES=0 # capture the "ln" commands in case the user needs to debug - ( - if [[ ${DO_MINI} == "false" ]]; then - # Doing daily, full timelapse - ls -rt "${DATE_DIR}"/*."${EXTENSION}" - exit 0 # Gets us out of this sub-shell - fi - - if [[ -f ${MINI_FILE} ]]; then - cat "${MINI_FILE}" - else - echo "${ME} WARNING: No '${MINI_FILE}' file" >&2 - # Do not pass anything to gawk - fi - ) | gawk -v DO_MINI=${DO_MINI} 'BEGIN { a=0; } - { - a++; - printf "ln -s %s '"${SEQUENCE_DIR}"'/%04d.'"${EXTENSION}"'\n", $0, a; - } - END { - # If we are in "mini" mode, tell bash to exit 1 so we do not have to create a temporary file. - if (a > 0 && DO_MINI == "true") { - printf("exit 1"); # avoids creating ${TMP} for MINI timelapse - } else if (a > 0) { - printf("Processed %d images\n", a) > "'"${TMP}"'"; - printf("exit 0"); - } else { # either a == 0 or in MINI mode - printf("exit 2"); # no, or not enough, images found - } + if [[ -n ${IMAGES_FILE} ]]; then + cat "${IMAGES_FILE}" - }' \ - | bash - RET=$? + # This is needed because NUM_IMAGES is updated in a sub-shell + # so we can't access it and hence don't know how many images were processed, + # and it's too expensive to count the number in SEQUENCE_DIR since it could + # have thousands of images. + echo "[end]" # signals end of the list + else + ls -rt "${INPUT_DIR}"/*."${EXTENSION}" 2>/dev/null + echo "[end]" + fi | while read -r IMAGE + do + if [[ ${IMAGE} == "[end]" ]]; then + if [[ ${NUM_IMAGES} -eq 0 ]]; then + exit 1 # gets out of "while" loop + elif [[ ${IS_MINI} == "false" ]]; then + echo "Processed ${NUM_IMAGES} images" > "${TMP}" + fi + else + # Make sure the file exists. + # This user or something else may have removed it. + if [[ ! -s ${IMAGE} ]]; then + if [[ ! -f ${IMAGE} ]]; then +# TODO: would be nice to remove from the file, +# but we don't create/update the file so any change we make may be overwritten. + MSG="not found" + else + MSG="has nothing in it" + rm -f "${IMAGE}" + fi + echo -e "${YELLOW}*** ${ME} WARNING: image '${IMAGE}' ${MSG}!${NC}" + continue + fi - # If bash exited with 0 then there are images and we're not in MINI mode. - # If bash exited with 1 we're in MINI mode; we exit 1 in MINI mode to avoid - # If bash exited with 2 no images were found. - # In MINI mode that's ok (but exit with 1 so the invoker knows we didn't create a timelapse). - if [[ ${RET} -eq 2 ]]; then - if [[ ${DO_MINI} == "false" ]]; then - echo -e "${RED}*** ${ME} ERROR: No images found!${NC}" - rm -fr "${SEQUENCE_DIR}" - fi + ((NUM_IMAGES++)) + NUM="$( printf "%04d" "${NUM_IMAGES}" )" + ln -s "${IMAGE}" "${SEQUENCE_DIR}/${NUM}.${EXTENSION}" + fi + done + if [[ $? -ne 0 ]]; then + echo -e "${RED}*** ${ME} ERROR: No images found in '${INPUT_DIR}'!${NC}" + rm -fr "${SEQUENCE_DIR}" + [[ -n ${PID_FILE} ]] && rm -f "${PID_FILE}" exit 1 fi else - echo -e "${ME} ${YELLOW}Not regenerating sequence because KEEP_SEQUENCE was given and ${NSEQ} links are present ${NC}" + echo -e "${ME} ${YELLOW}" + echo "Not regenerating sequence because KEEP_SEQUENCE is enabled." + echo -e "${NC}" fi SCALE="" @@ -210,7 +230,7 @@ SCALE="" # "-loglevel warning" gets rid of the dozens of lines of garbage output # but doesn't get rid of "deprecated pixel format" message when -pix_ftm is "yuv420p". # set FFLOG=info in config.sh if you want to see what's going on for debugging. -if [[ ${DO_MINI} == "true" ]]; then +if [[ ${IS_MINI} == "true" ]]; then FPS="${TIMELAPSE_MINI_FPS}" TIMELAPSE_BITRATE="${TIMELAPSE_MINI_BITRATE}" if [[ ${TIMELAPSE_MINI_WIDTH} != "0" ]]; then @@ -248,7 +268,7 @@ if [[ ${RET} -ne -0 ]]; then fi # if the user wants output, give it to them -[[ ${FFLOG} == "info" && ${DO_MINI} == "false" ]] && cat "${TMP}" +[[ ${FFLOG} == "info" && ${IS_MINI} == "false" ]] && cat "${TMP}" if [[ ${KEEP_SEQUENCE} == "false" ]]; then rm -rf "${SEQUENCE_DIR}" @@ -260,6 +280,7 @@ fi [[ ${DEBUG} -ge 2 ]] && echo -e "${ME}: ${GREEN}Timelapse in ${OUTPUT_FILE}${NC}" -rm -f "${PID_FILE}" +[[ -n ${PID_FILE} ]] && rm -f "${PID_FILE}" exit 0 + diff --git a/scripts/upload.sh b/scripts/upload.sh index 855338155..acc6ee31a 100755 --- a/scripts/upload.sh +++ b/scripts/upload.sh @@ -115,56 +115,24 @@ LOG="${ALLSKY_TMP}/upload_errors.txt" # Multiple concurrent uploads (which can happen if the system and/or network is slow can # cause errors and files left on the server. PID_FILE="${ALLSKY_TMP}/${FILE_TYPE}-pid.txt" -NUM_CHECKS=0 if [[ ${WAIT} == "true" ]]; then MAX_CHECKS=10 - SLEEP_TIME="5s" + SLEEP="5s" else MAX_CHECKS=2 - SLEEP_TIME="10s" + SLEEP="10s" +fi +ABORTED_MSG1="Another '${FILE_TYPE}' upload is in progress so the new upload of" +ABORTED_MSG1="${ABORTED_MSG1} $(basename "${FILE_TO_UPLOAD}") was aborted." +ABORTED_FIELDS="${FILE_TYPE}\t${FILE_TO_UPLOAD}" +ABORTED_MSG2="uploads" +CAUSED_BY="This could be caused network issues or by delays between images that are too short." +if ! one_instance --sleep "${SLEEP}" --max-checks "${MAX_CHECKS}" --pid-file "${PID_FILE}" \ + --aborted-count-file "${ALLSKY_ABORTEDUPLOADS}" --aborted-fields "${ABORTED_FIELDS}" \ + --aborted-msg1 "${ABORTED_MSG1}" --aborted-msg2 "${ABORTED_MSG2}" \ + --caused-by "${CAUSED_BY}" ; then + exit 1 fi -while : ; do - [[ ! -f ${PID_FILE} ]] && break - - PID=$( < "${PID_FILE}" ) - # shellcheck disable=SC2009 - if ! ps -p "${PID}" | grep --silent "${ME}" ; then - break # Not sure why the PID file existed if the process didn't exist. - fi - - if [[ $NUM_CHECKS -eq ${MAX_CHECKS} ]]; then - echo -en "${YELLOW}" >&2 - echo -n "${ME}: WARNING: Another '${FILE_TYPE}' upload is in" >&2 - echo " progress so the new upload of $(basename "${FILE_TO_UPLOAD}") was aborted." >&2 - echo -n "Made ${NUM_CHECKS} attempts at waiting." >&2 - echo -n " If this happens often, check your network and delay settings." >&2 - echo -e "${NC}" >&2 - ps -fp "${PID}" >&2 - - # Keep track of aborts so user can be notified. - # If it's happening often let the user know. - echo -e "$(date)\t${FILE_TYPE}\t${FILE_TO_UPLOAD}" >> "${ALLSKY_ABORTEDUPLOADS}" - NUM=$( wc -l < "${ALLSKY_ABORTEDUPLOADS}" ) - if [[ ${NUM} -eq 3 || ${NUM} -eq 10 ]]; then - MSG="${NUM} uploads have been aborted waiting for other uploads to finish." - MSG="${MSG}\nThis could be caused by a slow network or other network issues." - if [[ ${NUM} -eq 3 ]]; then - SEVERITY="info" - else - SEVERITY="warning" - MSG="${MSG}\nOnce you have resolved the cause, reset the aborted counter:" - MSG="${MSG}\n    rm -f '${ALLSKY_ABORTEDUPLOADS}'" - fi - "${ALLSKY_SCRIPTS}/addMessage.sh" "${SEVERITY}" "${MSG}" - fi - - exit 2 - else - sleep "${SLEEP_TIME}" - fi - ((NUM_CHECKS++)) -done -echo $$ > "${PID_FILE}" || exit 1 # Convert to lowercase so we don't care if user specified upper or lowercase. PROTOCOL="${PROTOCOL,,}" @@ -177,11 +145,10 @@ trap "" SIGTERM trap "" SIGHUP if [[ ${PROTOCOL} == "s3" ]] ; then - # xxxxxx How do you tell it the DESTINATION_NAME name ? if [[ ${SILENT} == "false" && ${ALLSKY_DEBUG_LEVEL} -ge 3 ]]; then - echo "${ME}: Uploading ${FILE_TO_UPLOAD} to aws ${S3_BUCKET}/${REMOTE_DIR}" + echo "${ME}: Uploading ${FILE_TO_UPLOAD} to aws ${S3_BUCKET}${REMOTE_DIR}/${DESTINATION_NAME}" fi - OUTPUT="$("${AWS_CLI_DIR}/aws" s3 cp "${FILE_TO_UPLOAD}" "s3://${S3_BUCKET}${REMOTE_DIR}" --acl "${S3_ACL}" 2>&1)" + OUTPUT="$("${AWS_CLI_DIR}/aws" s3 cp "${FILE_TO_UPLOAD}" "s3://${S3_BUCKET}${REMOTE_DIR}/${DESTINATION_NAME}" --acl "${S3_ACL}" 2>&1)" RET=$? @@ -196,17 +163,17 @@ elif [[ ${PROTOCOL} == "local" ]] ; then elif [[ "${PROTOCOL}" == "scp" ]] ; then if [[ ${SILENT} == "false" && ${ALLSKY_DEBUG_LEVEL} -ge 3 ]]; then # shellcheck disable=SC2153 - echo "${ME}: Copying ${FILE_TO_UPLOAD} to ${REMOTE_HOST}:${REMOTE_DIR}/${DESTINATION_NAME}" + echo "${ME}: Copying ${FILE_TO_UPLOAD} to ${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_DIR}/${DESTINATION_NAME}" fi [[ -n ${REMOTE_PORT} ]] && REMOTE_PORT="-P ${REMOTE_PORT}" # shellcheck disable=SC2086 - OUTPUT="$(scp -i "${SSH_KEY_FILE}" ${REMOTE_PORT} "${FILE_TO_UPLOAD}" "${REMOTE_HOST}:${REMOTE_DIR}/${DESTINATION_NAME}" 2>&1)" + OUTPUT="$(scp -i "${SSH_KEY_FILE}" ${REMOTE_PORT} "${FILE_TO_UPLOAD}" "${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_DIR}/${DESTINATION_NAME}" 2>&1)" RET=$? elif [[ ${PROTOCOL} == "gcs" ]] ; then if [[ ${SILENT} == "false" && ${ALLSKY_DEBUG_LEVEL} -ge 3 ]]; then - echo "${ME}: Uploading ${FILE_TO_UPLOAD} to gcs ${GCS_BUCKET}/${REMOTE_DIR}" + echo "${ME}: Uploading ${FILE_TO_UPLOAD} to gcs ${GCS_BUCKET}${REMOTE_DIR}" fi OUTPUT="$(gsutil cp -a "${GCS_ACL}" "${FILE_TO_UPLOAD}" "gs://${GCS_BUCKET}${REMOTE_DIR}" 2>&1)" RET=$? @@ -259,10 +226,12 @@ else # sftp/ftp/ftps # lftp doesn't actually try to open the connection until the first command is executed, # and if it fails the error message isn't always clear. # So, do a simple command first so we get a better error message. - echo "quote PWD > /dev/null || cd . || exit 99" # PWD not supported by all servers + echo "cd . || exit 99" if [[ ${DEBUG} == "true" ]]; then - echo "quote PWD" + # PWD not supported by all servers, + # but if it works it returns "xxx is current directory" so only output that. + echo "quote PWD | grep current " echo "ls" echo "debug 5" fi @@ -330,7 +299,7 @@ else # sftp/ftp/ftps echo -e "\n${YELLOW}Commands used${NC} are in: ${GREEN}${LFTP_CMDS}${NC}" else - if [[ ${SILENT} == "false" && ${ALLSKY_DEBUG_LEVEL} -ge 3 && ${ON_TTY} -eq 0 ]]; then + if [[ ${SILENT} == "false" && ${ALLSKY_DEBUG_LEVEL} -ge 4 && ${ON_TTY} -eq 0 ]]; then echo "${ME}: FTP '${FILE_TO_UPLOAD}' finished" fi fi diff --git a/src/ASI_functions.cpp b/src/ASI_functions.cpp index 480bd8a44..aa288b592 100644 --- a/src/ASI_functions.cpp +++ b/src/ASI_functions.cpp @@ -128,21 +128,30 @@ ASI_CAMERA_INFO ASICameraInfoArray[] = { "imx477", 0, "RPi HQ", 0, 3040, 4056, ASI_TRUE, // Need ASI_IMG_END so we know where the end of the list is. BAYER_RG, {1, 2, 0}, {ASI_IMG_RGB24, ASI_IMG_END}, 1.55, ASI_FALSE, - 12, ASI_FALSE, ASI_FALSE}, + 12, ASI_FALSE, ASI_FALSE + }, // There are many versions of the imx708 (_wide, _noir, _wide_noir, etc.) // so just check for "imx708" (6 characters. { "imx708", 6, "RPi Module 3", 0, 2592, 4608, ASI_TRUE, BAYER_RG, {1, 2, 0}, {ASI_IMG_RGB24, ASI_IMG_END}, 1.40, ASI_FALSE, - 10, ASI_FALSE, ASI_TRUE}, + 10, ASI_FALSE, ASI_TRUE + }, + + { "imx290", 0, "imx290 60.00 fps", 0, 1920, 1080, ASI_TRUE, + BAYER_RG, {1, 2, 0}, {ASI_IMG_RGB24, ASI_IMG_END}, 2.9, ASI_FALSE, + 12, ASI_FALSE, ASI_FALSE + }, { "imx519", 0, "ArduCam 16 MP", 0, 3496, 4656, ASI_TRUE, BAYER_RG, {1, 2, 0}, {ASI_IMG_RGB24, ASI_IMG_END}, 1.22, ASI_FALSE, - 10, ASI_FALSE, ASI_TRUE}, + 10, ASI_FALSE, ASI_TRUE + }, { "arducam_64mp", 0, "ArduCam 64 MP", 0, 6944, 9152, ASI_TRUE, BAYER_RG, {1, 2, 0}, {ASI_IMG_RGB24, ASI_IMG_END}, 0.8, ASI_FALSE, - 10, ASI_FALSE, ASI_TRUE}, + 10, ASI_FALSE, ASI_TRUE + }, // FUTURE CAMERAS GO HERE... }; @@ -195,7 +204,7 @@ ASI_CONTROL_CAPS ControlCapsArray[][MAX_NUM_CONTROL_CAPS] = // 99 == don't know // Name, Description, MaxValue, MinValue, DefaultValue, CurrentValue, IsAutoSupported, IsWritable, ControlType - { // imx477, libcamera + { // imx477, libcamera THIS MUST BE THE FIRST CAMERA { "Gain", "Gain", 16.0, 1, 1, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_GAIN }, { "Exposure", "Exposure Time (us)", 230 * US_IN_SEC, 1, 10000, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_EXPOSURE }, { "WB_R", "White balance: Red component", 10.0, 0.1, 2.5, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_WB_R }, @@ -253,6 +262,27 @@ ASI_CONTROL_CAPS ControlCapsArray[][MAX_NUM_CONTROL_CAPS] = { "End", "End", 0.0, 0.0, 0.0, 0.0, ASI_FALSE, ASI_FALSE, CONTROL_TYPE_END }, }, + { // imx290, libcamera + { "Gain", "Gain", 16.0, 1, 1, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_GAIN }, + { "Exposure", "Exposure Time (us)", 200 * US_IN_SEC, 1, 10000, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_EXPOSURE }, + { "WB_R", "White balance: Red component", 10.0, 0.1, 2.5, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_WB_R }, + { "WB_B", "White balance: Blue component", 10.0, 0.1, 2.0, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_WB_B }, + { "Temperature", "Sensor Temperature", 80, -20, NOT_SET, NOT_SET, ASI_FALSE, ASI_FALSE, ASI_TEMPERATURE }, + { "Flip", "Flip: 0->None, 1->Horiz, 2->Vert, 3->Both", 3, 0, 0, NOT_SET, ASI_FALSE, ASI_TRUE, ASI_FLIP }, + { "AutoExpMaxGain", "Auto exposure maximum gain value", 16.0, 1, 16.0, NOT_SET, ASI_FALSE, ASI_TRUE, ASI_AUTO_MAX_GAIN }, + { "AutoExpMaxExpMS", "Auto exposure maximum exposure value (ms)", 200 * MS_IN_SEC, 1, 60 * MS_IN_SEC, NOT_SET, ASI_FALSE, ASI_TRUE, ASI_AUTO_MAX_EXP }, + { "ExposureCompensation", "Exposure Compensation", 10.0, -10.0, 0, NOT_SET, ASI_FALSE, ASI_TRUE, EV }, + { "Brightness", "Brightness", 1.0, -1.0, 0, NOT_SET, ASI_FALSE, ASI_TRUE, ASI_AUTO_TARGET_BRIGHTNESS }, + { "Saturation", "Saturation", 15.99, 0.0, 1.0, NOT_SET, ASI_FALSE, ASI_TRUE, SATURATION }, + { "Contrast", "Contrast", 15.99, 0.0, 1.0, NOT_SET, ASI_FALSE, ASI_TRUE, CONTRAST }, + { "Sharpness", "Sharpness", 15.99, 0.0, 1.0, NOT_SET, ASI_FALSE, ASI_TRUE, SHARPNESS }, + + { "End", "End", 0.0, 0.0, 0.0, 0.0, ASI_FALSE, ASI_FALSE, CONTROL_TYPE_END }, + }, + { // imx290, raspistill. Not supported. + { "End", "End", 0.0, 0.0, 0.0, 0.0, ASI_FALSE, ASI_FALSE, CONTROL_TYPE_END }, + }, + { // imx519, libcamera { "Gain", "Gain", 16.0, 1, 1, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_GAIN }, { "Exposure", "Exposure Time (us)", 200 * US_IN_SEC, 1, 10000, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_EXPOSURE }, @@ -276,7 +306,6 @@ ASI_CONTROL_CAPS ControlCapsArray[][MAX_NUM_CONTROL_CAPS] = { // arducam_64mp, libcamera - // TODO: Update as necessary { "Gain", "Gain", 16.0, 1, 1, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_GAIN }, { "Exposure", "Exposure Time (us)", 200 * US_IN_SEC, 1, 10000, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_EXPOSURE }, { "WB_R", "White balance: Red component", 10.0, 0.1, 2.5, NOT_SET, ASI_TRUE, ASI_TRUE, ASI_WB_R }, @@ -296,6 +325,8 @@ ASI_CONTROL_CAPS ControlCapsArray[][MAX_NUM_CONTROL_CAPS] = { // arducam_64mp, raspistill. Not supported. { "End", "End", 0.0, 0.0, 0.0, 0.0, ASI_FALSE, ASI_FALSE, CONTROL_TYPE_END }, }, + + // FUTURE CAMERAS GO HERE... }; char camerasInfoFile[128] = { 0 }; // name of temporary file @@ -354,33 +385,32 @@ int getCameraNumber() // Determine which camera sensor(s) we have by reading the file created in ASIGetNumOfConnectedCameras(). if (camerasInfoFile[0] == '\0') { - Log(0, "ERROR: camerasInfoFile not created!\n"); + Log(0, "%s: ERROR: camerasInfoFile not created!\n", CG.ME); closeUp(EXIT_ERROR_STOP); } FILE *f = fopen(camerasInfoFile, "r"); if (f == NULL) { - Log(0, "ERROR: Unable to open '%s': %s\n", camerasInfoFile, strerror(errno)); + Log(0, "%s: ERROR: Unable to open '%s': %s\n", CG.ME, camerasInfoFile, strerror(errno)); closeUp(EXIT_ERROR_STOP); } char line[256]; int num = NOT_SET; #define SENSOR_STRING_SIZE 25 - char sensor[SENSOR_STRING_SIZE], found_sensor[SENSOR_STRING_SIZE] = { 0 }; - bool found = false; - int actualIndex; // index into ASICameraInfoArray[] - int RPiCameraIndex = 0; // index into RPiCameras[] + char sensor[SENSOR_STRING_SIZE]; + int actualIndex; // index into ASICameraInfoArray[] + int RPiCameraIndex = -1; // index into RPiCameras[] // For each camera found, update the next *RPiCameras[] entry to point to the // camera's ASICameraInfoArray[] entry. // Return the index into *RPiCameras[] of the attached camera we're using. - while (fgets(line, sizeof(line), f) != NULL) + while (fgets(line, sizeof(line)-1, f) != NULL) { + // Sample line: 0 : imx477 [4056x3040] .... + // We only care about first two. if (sscanf(line, "%d : %s ", &num, sensor) == 2) { - strncpy(found_sensor, sensor, SENSOR_STRING_SIZE); // last sensor found - // Found a camera; check all known cameras to make sure it's one we know about. // Unfortunately we don't have anything else to check, like serial number. // I suppose we could also check the Modes are the same, but it's not worth it. @@ -399,8 +429,8 @@ int getCameraNumber() { // The sensor is in our list. foundThisSensor = true; - found = true; - actualIndex = (int) i; + actualIndex = i; + RPiCameraIndex++; RPiCameras[RPiCameraIndex].CameraInfo = &ASICameraInfoArray[actualIndex]; // There are TWO entries in ControlCapsArray[] for every entry in ASICameraInfoArray[]. @@ -410,25 +440,23 @@ int getCameraNumber() actualIndex = (actualIndex * 2) + (CG.isLibcamera ? 0 : 1); RPiCameras[RPiCameraIndex].ControlCaps = &ControlCapsArray[actualIndex][0]; Log(4, " ControlCapsArray[%d].\n", actualIndex); - RPiCameraIndex++; break; } } if (! foundThisSensor) { - Log(1, "WARNING: Sensor '%s' connected to Pi but not known by Allsky\n", sensor); + Log(1, "%s: WARNING: Sensor '%s' found but not supported by Allsky.\n", CG.ME, sensor); } } } - if (! found) + if (RPiCameraIndex == -1) { - Log(0, "ERROR: Could not find information on camera '%s'.\n", - *found_sensor ? found_sensor : "unknown sensor"); - closeUp(EXIT_ERROR_STOP); + Log(0, "%s: ERROR: No RPi cameras found.\n", CG.ME); + closeUp(EXIT_NO_CAMERA); } // Caller will determine if this is out of range. - return(CG.cameraNumber); + return(RPiCameraIndex); } // Put the properties for the specified camera into pASICameraInfo. @@ -436,7 +464,7 @@ ASI_ERROR_CODE ASIGetCameraProperty(ASI_CAMERA_INFO *pASICameraInfo, int iCamera { if (iCameraIndex < 0 || iCameraIndex >= CG.numCameras) { - Log(0, "ERROR: ASIGetCameraProperty(), iCameraIndex (%d) bad.\n", iCameraIndex); + Log(0, "%s: ERROR: ASIGetCameraProperty(), iCameraIndex (%d) bad.\n", CG.ME, iCameraIndex); return(ASI_ERROR_INVALID_INDEX); } @@ -450,7 +478,7 @@ ASI_ERROR_CODE ASIGetNumOfControls(int iCameraIndex, int *piNumberOfControls) { if (iCameraIndex < 0 || iCameraIndex >= CG.numCameras) { - Log(0, "ERROR: ASIGetNumOfControls(), iCameraIndex (%d) bad.\n", iCameraIndex); + Log(0, "%s: ERROR: ASIGetNumOfControls(), iCameraIndex (%d) bad.\n", CG.ME, iCameraIndex); return(ASI_ERROR_INVALID_INDEX); } @@ -647,14 +675,15 @@ void processConnectedCameras() CG.numCameras = ASIGetNumOfConnectedCameras(); if (CG.numCameras <= 0) { - Log(0, "*** ERROR: No Connected Camera...\n"); + Log(0, "*** %s: ERROR: No Connected Camera...\n", CG.ME); closeUp(EXIT_NO_CAMERA); // If there are no cameras we can't do anything. } CG.cameraNumber = getCameraNumber(); if (CG.cameraNumber >= CG.numCameras) { - Log(0, "*** ERROR: Camera number %d not connected. Highest number is %d.\n", CG.cameraNumber, CG.numCameras-1); + Log(0, "*** %s: ERROR: Camera number %d not connected. Highest number is %d.\n", + CG.ME, CG.cameraNumber, CG.numCameras-1); closeUp(EXIT_NO_CAMERA); } else if (CG.numCameras > 1) @@ -685,7 +714,8 @@ ASI_ERROR_CODE getControlCapForControlType(int iCameraIndex, ASI_CONTROL_TYPE Co ASI_ERROR_CODE ret; ret = ASIGetControlCaps(iCameraIndex, i, &cc); if (ret != ASI_SUCCESS) { - Log(3, "ASIGetControlCaps(%d, %i, &cc) failed: %s\n", iCameraIndex, i, getRetCode(ret)); + Log(3, "%s: ASIGetControlCaps(%d, %i, &cc) failed: %s\n", + CG.ME, iCameraIndex, i, getRetCode(ret)); return(ret); } if (ControlType == cc.ControlType) @@ -713,7 +743,8 @@ char *getSerialNumber(int camNum) if (asiRetCode == ASI_ERROR_GENERAL_ERROR) snprintf(sn, sizeof(sn), "[not supported]"); else - Log(1, "*** WARNING: unable to get camera serial number (%s)\n", getRetCode(asiRetCode)); + Log(1, "*** %s: WARNING: unable to get camera serial number (%s)\n", + CG.ME, getRetCode(asiRetCode)); } else if (serialNumber.id[0] == '\0') { @@ -765,7 +796,7 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int FILE *f = fopen(file, "w"); if (f == NULL) { - Log(0, "ERROR: Unable to open '%s': %s\n", file, strerror(errno)); + Log(0, "%s: ERROR: Unable to open '%s': %s\n", CG.ME, file, strerror(errno)); closeUp(EXIT_ERROR_STOP); } Log(4, "saveCameraInfo(): saving to %s\n", file); @@ -883,6 +914,47 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int fprintf(f, "\t\t\t\"MaxValue\" : \"%s\"\n", "none"); fprintf(f, "\t\t},\n"); + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "daymean"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "daymean"); + fprintf(f, "\t\t\t\"MinValue\" : 0.0,\n"); + fprintf(f, "\t\t\t\"MaxValue\" : 1.0,\n"); + fprintf(f, "\t\t\t\"DefaultValue\" : %.2f\n", CG.myModeMeanSetting.dayMean); + fprintf(f, "\t\t},\n"); + + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "DayMeanThreshold"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "daymeanthreshold"); + fprintf(f, "\t\t\t\"MinValue\" : 0.01,\n"); + fprintf(f, "\t\t\t\"MaxValue\" : \"none\",\n"); + fprintf(f, "\t\t\t\"DefaultValue\" : %f\n", CG.myModeMeanSetting.dayMean_threshold); + fprintf(f, "\t\t},\n"); + + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "nightmean"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "nightmean"); + fprintf(f, "\t\t\t\"MinValue\" : 0.0,\n"); + fprintf(f, "\t\t\t\"MaxValue\" : 1.0,\n"); + fprintf(f, "\t\t\t\"DefaultValue\" : %.2f\n", CG.myModeMeanSetting.nightMean); + fprintf(f, "\t\t},\n"); + + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "NightMeanThreshold"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "nightmeanthreshold"); + fprintf(f, "\t\t\t\"MinValue\" : 0.01,\n"); + fprintf(f, "\t\t\t\"MaxValue\" : \"none\",\n"); + fprintf(f, "\t\t\t\"DefaultValue\" : %f\n", CG.myModeMeanSetting.nightMean_threshold); + fprintf(f, "\t\t},\n"); + +// TODO: remove in next release +fprintf(f, "\t\t{\n"); +fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "MeanThreshold"); +fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "meanthreshold"); +fprintf(f, "\t\t\t\"MinValue\" : 0.01,\n"); +fprintf(f, "\t\t\t\"MaxValue\" : \"none\",\n"); +fprintf(f, "\t\t\t\"DefaultValue\" : %f\n", CG.myModeMeanSetting.mean_threshold); +fprintf(f, "\t\t},\n"); + if (CG.isColorCamera) { fprintf(f, "\t\t{\n"); fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "AutoWhiteBalance"); @@ -921,6 +993,7 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int fprintf(f, "\t\t\t\"DefaultValue\" : %ld\n", CG.gainTransitionTime); fprintf(f, "\t\t},\n"); } + #ifdef IS_ZWO fprintf(f, "\t\t{\n"); fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "autousb"); @@ -934,6 +1007,12 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int fprintf(f, "\t\t\t\"DefaultValue\" : %d\n", CG.overlay.showUSB ? 1 : 0); fprintf(f, "\t\t},\n"); + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "experimentalExposure"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "experimentalExposure"); + fprintf(f, "\t\t\t\"DefaultValue\" : \"%d\"\n", CG.HB.useExperimentalExposure ? 1 : 0); + fprintf(f, "\t\t},\n"); + fprintf(f, "\t\t{\n"); fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "histogrambox"); fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "histogrambox"); @@ -953,8 +1032,9 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int fprintf(f, "\t\t},\n"); fprintf(f, "\t\t{\n"); - fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "cameraNumber"); - fprintf(f, "\t\t\t\"argumentName\" : \"%s\"\n", "cameraNumber"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "CameraNumber"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "cameraNumber"); + fprintf(f, "\t\t\t\"DefaultValue\" : %d\n", CG.cameraNumber); fprintf(f, "\t\t},\n"); #endif @@ -964,23 +1044,6 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int fprintf(f, "\t\t\t\"argumentName\" : \"%s\"\n", "extraArgs"); fprintf(f, "\t\t},\n"); - fprintf(f, "\t\t{\n"); - fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "ModeMean"); - fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "mean"); - fprintf(f, "\t\t\t\"MinValue\" : 0.0,\n"); - fprintf(f, "\t\t\t\"MaxValue\" : 1.0,\n"); - fprintf(f, "\t\t\t\"DefaultValue\" : \"day: %.2f, night: %.2f\"\n", - CG.myModeMeanSetting.dayMean, CG.myModeMeanSetting.nightMean); - fprintf(f, "\t\t},\n"); - - fprintf(f, "\t\t{\n"); - fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "MeanThreshold"); - fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "meanthreshold"); - fprintf(f, "\t\t\t\"MinValue\" : 0.01,\n"); - fprintf(f, "\t\t\t\"MaxValue\" : \"none\",\n"); - fprintf(f, "\t\t\t\"DefaultValue\" : %f\n", CG.myModeMeanSetting.mean_threshold); - fprintf(f, "\t\t},\n"); - if (CG.ct == ctRPi && CG.isLibcamera) { fprintf(f, "\t\t{\n"); fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "TuningFile"); @@ -990,6 +1053,8 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int } #endif + double minGain=0.0, maxGain=0.0; + for (int i = 0; i < iNumOfCtrl; i++) { ASI_CONTROL_CAPS cc; @@ -1004,26 +1069,72 @@ void saveCameraInfo(ASI_CAMERA_INFO cameraInfo, char const *file, int width, int if (a == NULL or a[0] == '\0') continue; - if (i > 0) - { - fprintf(f, ","); // comma on all but last one - fprintf(f, "\n"); + int div_by = 1; + if (strcmp(cc.Name, "Exposure") == 0) { + // The camera's values are in microseconds (us), but the WebUI displays in milliseconds (ms). + div_by = US_IN_MS; } + double min = cc.MinValue / (double) div_by; + double max = cc.MaxValue / (double) div_by; + double def = cc.DefaultValue / (double) div_by; fprintf(f, "\t\t{\n"); fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", cc.Name); fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", a); - fprintf(f, "\t\t\t\"MinValue\" : %s,\n", LorF(cc.MinValue, "%ld", "%.3f")); - fprintf(f, "\t\t\t\"MaxValue\" : %s,\n", LorF(cc.MaxValue, "%ld", "%.3f")); - fprintf(f, "\t\t\t\"DefaultValue\" : %s,\n", LorF(cc.DefaultValue, "%ld", "%.3f")); + fprintf(f, "\t\t\t\"MinValue\" : %s,\n", LorF(min, "%ld", "%.3f")); + fprintf(f, "\t\t\t\"MaxValue\" : %s,\n", LorF(max, "%ld", "%.3f")); + fprintf(f, "\t\t\t\"DefaultValue\" : %s,\n", LorF(def, "%ld", "%.3f")); fprintf(f, "\t\t\t\"IsAutoSupported\" : %s,\n", cc.IsAutoSupported == ASI_TRUE ? "true" : "false"); fprintf(f, "\t\t\t\"IsWritable\" : %s,\n", cc.IsWritable == ASI_TRUE ? "true" : "false"); fprintf(f, "\t\t\t\"ControlType\" : %d\n", cc.ControlType); - fprintf(f, "\t\t}"); + fprintf(f, "\t\t},\n"); + + if (strcmp(cc.Name, "Gain") == 0) { + minGain = cc.MinValue; + maxGain = cc.MaxValue; + } } - fprintf(f, "\n"); - fprintf(f, "\t]\n"); + // These override the generic values. + + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "dayexposure"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "dayexposure"); + fprintf(f, "\t\t\t\"DefaultValue\" : %d\n", (int) (0.5 * MS_IN_SEC)); + fprintf(f, "\t\t},\n"); + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "nightexposure"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "nightexposure"); + fprintf(f, "\t\t\t\"DefaultValue\" : %d\n", 10 * MS_IN_SEC); + fprintf(f, "\t\t},\n"); + + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "daymean"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "daymean"); + fprintf(f, "\t\t\t\"DefaultValue\" : %f\n", CG.myModeMeanSetting.dayMean); + fprintf(f, "\t\t},\n"); + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "nightmean"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "nightmean"); + fprintf(f, "\t\t\t\"DefaultValue\" : %f\n", CG.myModeMeanSetting.nightMean); + fprintf(f, "\t\t},\n"); + + // Set the day gain to the minimum possible. + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "daygain"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "daygain"); + fprintf(f, "\t\t\t\"DefaultValue\" : %s\n", LorF(minGain, "%ld", "%.3f")); + fprintf(f, "\t\t},\n"); + // Set the night gain to the half the max. + fprintf(f, "\t\t{\n"); + fprintf(f, "\t\t\t\"Name\" : \"%s\",\n", "nightgain"); + fprintf(f, "\t\t\t\"argumentName\" : \"%s\",\n", "nightgain"); + fprintf(f, "\t\t\t\"DefaultValue\" : %s\n", LorF(maxGain / 2, "%ld", "%.3f")); + fprintf(f, "\t\t}\n"); // NO COMMA ON LAST ONE + + + // End the list + fprintf(f, "\t]\n"); fprintf(f, "}\n"); fclose(f); } @@ -1137,33 +1248,39 @@ bool checkExposureValues(config *cg) { if (cg->dayExposure_us < cg->cameraMinExposure_us) { - Log(1, "*** WARNING: daytime exposure %'ld us less than camera minimum of %'ld us; setting to minimum\n", cg->dayExposure_us, cg->cameraMinExposure_us); + Log(1, "*** %s: WARNING: daytime exposure %'ld us less than camera minimum of %'ld us; setting to minimum\n", + cg->ME, cg->dayExposure_us, cg->cameraMinExposure_us); cg->dayExposure_us = cg->cameraMinExposure_us; } else if (cg->dayExposure_us > cg->cameraMaxExposure_us) { - Log(1, "*** WARNING: daytime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", cg->dayExposure_us, cg->cameraMaxExposure_us); + Log(1, "*** %s: WARNING: daytime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", + cg->ME, cg->dayExposure_us, cg->cameraMaxExposure_us); cg->dayExposure_us = cg->cameraMaxExposure_us; } else if (cg->dayAutoExposure && cg->dayExposure_us > cg->cameraMaxAutoExposure_us) { - Log(1, "*** WARNING: daytime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", cg->dayExposure_us, cg->cameraMaxAutoExposure_us); + Log(1, "*** %s: WARNING: daytime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", + cg->ME, cg->dayExposure_us, cg->cameraMaxAutoExposure_us); cg->dayExposure_us = cg->cameraMaxAutoExposure_us; } if (cg->nightExposure_us < cg->cameraMinExposure_us) { - Log(1, "*** WARNING: nighttime exposure %'ld us less than camera minimum of %'ld us; setting to minimum\n", cg->nightExposure_us, cg->cameraMinExposure_us); + Log(1, "*** %s: WARNING: nighttime exposure %'ld us less than camera minimum of %'ld us; setting to minimum\n", + cg->ME, cg->nightExposure_us, cg->cameraMinExposure_us); cg->nightExposure_us = cg->cameraMinExposure_us; } else if (cg->nightExposure_us > cg->cameraMaxExposure_us) { - Log(1, "*** WARNING: nighttime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", cg->nightExposure_us, cg->cameraMaxExposure_us); + Log(1, "*** %s: WARNING: nighttime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", + cg->ME, cg->nightExposure_us, cg->cameraMaxExposure_us); cg->nightExposure_us = cg->cameraMaxExposure_us; } else if (cg->nightAutoExposure && cg->nightExposure_us > cg->cameraMaxAutoExposure_us) { - Log(1, "*** WARNING: nighttime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", cg->nightExposure_us, cg->cameraMaxAutoExposure_us); + Log(1, "*** %s: WARNING: nighttime exposure %'ld us greater than camera maximum of %'ld us; setting to maximum\n", + cg->ME, cg->nightExposure_us, cg->cameraMaxAutoExposure_us); cg->nightExposure_us = cg->cameraMaxAutoExposure_us; } @@ -1185,7 +1302,8 @@ static bool checkBin(long b, ASI_CAMERA_INFO ci, char const *field) break; } if (! ok) - Log(0, "*** ERROR: %s bin of %ldx%ld not supported by camera %s.\n", field, b, b, ci.Name); + Log(0, "*** %s: ERROR: %s bin of %ldx%ld not supported by camera %s.\n", + CG.ME, field, b, b, ci.Name); return(ok); } @@ -1217,8 +1335,23 @@ bool setDefaults(config *cg, ASI_CAMERA_INFO ci) cg->supportsMyModeMean = false; cg->gainTransitionTimeImplemented = true; cg->imagesSavedInBackground = true; + cg->myModeMeanSetting.dayMean = DEFAULT_DAYMEAN_ZWO; + cg->myModeMeanSetting.nightMean = DEFAULT_NIGHTMEAN_ZWO; + cg->myModeMeanSetting.minMean = DEFAULT_MINMEAN_ZWO; // min number a user should enter + cg->myModeMeanSetting.maxMean = DEFAULT_MAXMEAN_ZWO; // max number a user should enter + cg->myModeMeanSetting.dayMean_threshold = DEFAULT_DAYMEAN_THRESHOLD_ZWO; + cg->myModeMeanSetting.nightMean_threshold = DEFAULT_NIGHTMEAN_THRESHOLD_ZWO; + cg->myModeMeanSetting.minMean_threshold = DEFAULT_MINMEAN_THRESHOLD_ZWO; + cg->myModeMeanSetting.maxMean_threshold = DEFAULT_MAXMEAN_THRESHOLD_ZWO; + cg->myModeMeanSetting.mean_threshold = DEFAULT_DAYMEAN_THRESHOLD_ZWO; // TODO: xxxx remove in next release + cg->myModeMeanSetting.mean_p0 = DEFAULT_MEAN_P0_ZWO; + cg->myModeMeanSetting.mean_p1 = DEFAULT_MEAN_P1_ZWO; + cg->myModeMeanSetting.mean_p2 = DEFAULT_MEAN_P2_ZWO; + cg->myModeMeanSetting.minMean_p = DEFAULT_MINMEAN_P_ZWO; + cg->myModeMeanSetting.maxMean_p = DEFAULT_MAXMEAN_P_ZWO; + } else { // RPi -#ifdef IS_RPi +#ifdef IS_RPi // need this so it compiles cg->supportsTemperature = ci.SupportsTemperature; // this field only exists in RPi structure cg->divideTemperatureBy = 1.0; cg->supportsAutoFocus = ci.SupportsAutoFocus; // this field only exists in RPi structure @@ -1227,6 +1360,20 @@ bool setDefaults(config *cg, ASI_CAMERA_INFO ci) cg->supportsMyModeMean = true; cg->gainTransitionTimeImplemented = false; cg->imagesSavedInBackground = false; + cg->myModeMeanSetting.dayMean = DEFAULT_DAYMEAN_RPi; + cg->myModeMeanSetting.nightMean = DEFAULT_NIGHTMEAN_RPi; + cg->myModeMeanSetting.minMean = DEFAULT_MINMEAN_RPi; + cg->myModeMeanSetting.maxMean = DEFAULT_MAXMEAN_RPi; + cg->myModeMeanSetting.dayMean_threshold = DEFAULT_DAYMEAN_THRESHOLD_RPi; + cg->myModeMeanSetting.nightMean_threshold = DEFAULT_NIGHTMEAN_THRESHOLD_RPi; + cg->myModeMeanSetting.minMean_threshold = DEFAULT_MINMEAN_THRESHOLD_RPi; + cg->myModeMeanSetting.maxMean_threshold = DEFAULT_MAXMEAN_THRESHOLD_RPi; + cg->myModeMeanSetting.mean_threshold = DEFAULT_DAYMEAN_THRESHOLD_RPi; // TODO: xxxx delete in next release + cg->myModeMeanSetting.mean_p0 = DEFAULT_MEAN_P0_RPi; + cg->myModeMeanSetting.mean_p1 = DEFAULT_MEAN_P1_RPi; + cg->myModeMeanSetting.mean_p2 = DEFAULT_MEAN_P2_RPi; + cg->myModeMeanSetting.minMean_p = DEFAULT_MINMEAN_P_RPi; + cg->myModeMeanSetting.maxMean_p = DEFAULT_MAXMEAN_P_RPi; } if (cg->imagesSavedInBackground) { @@ -1237,6 +1384,18 @@ bool setDefaults(config *cg, ASI_CAMERA_INFO ci) cg->minDelay_ms = 0; } + // The remaining settings are camera-specific and have camera defaults. + ret = getControlCapForControlType(cg->cameraNumber, ASI_AUTO_TARGET_BRIGHTNESS, &cc); + if (ret == ASI_SUCCESS) + { + cg->defaultBrightness = cc.DefaultValue; // used elsewhere + cg->dayBrightness = cc.DefaultValue; + cg->nightBrightness = cc.DefaultValue; + } else { + Log(0, "%s: ASI_EXPOSURE failed with %s\n", cg->ME, getRetCode(ret)); + ok = false; + } + // Get values used in several validations. ret = getControlCapForControlType(cg->cameraNumber, ASI_EXPOSURE, &cc); if (ret == ASI_SUCCESS) @@ -1249,16 +1408,16 @@ bool setDefaults(config *cg, ASI_CAMERA_INFO ci) { cg->cameraMaxAutoExposure_us = cc.MaxValue * US_IN_MS; } else { - Log(0, "ASI_AUTO_MAX_EXP failed with %s\n", getRetCode(ret)); + Log(0, "%s: ASI_AUTO_MAX_EXP failed with %s\n", cg->ME, getRetCode(ret)); ok = false; - } + } } else { - Log(0, "ASI_EXPOSURE failed with %s\n", getRetCode(ret)); + Log(0, "%s: ASI_EXPOSURE failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } - signal(SIGINT, IntHandle); - signal(SIGTERM, IntHandle); // The service sends SIGTERM to end this program. + signal(SIGINT, IntHandle); // When run at the command line, this signal terminates us. + signal(SIGTERM, IntHandle); // The service sends SIGTERM to end this program. signal(SIGHUP, IntHandle); // SIGHUP means restart. return(ok); @@ -1303,11 +1462,19 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) cg->nightExposure_us = cg->temp_nightExposure_ms * US_IN_MS; cg->nightMaxAutoExposure_us = cg->temp_nightMaxAutoExposure_ms * US_IN_MS; - if (! validateFloat(&cg->myModeMeanSetting.dayMean, 0.0, 1.0, "Daytime Mean Target", false)) + if (! validateFloat(&cg->myModeMeanSetting.dayMean, cg->myModeMeanSetting.minMean, cg->myModeMeanSetting.maxMean, "Daytime Mean Target", false)) + ok = false; + if (! validateFloat(&cg->myModeMeanSetting.dayMean_threshold, cg->myModeMeanSetting.minMean_threshold, cg->myModeMeanSetting.maxMean_threshold, "Mean Threshold", false)) ok = false; - if (! validateFloat(&cg->myModeMeanSetting.nightMean, 0.0, 1.0, "Nighttime Mean Target", false)) + if (! validateFloat(&cg->myModeMeanSetting.nightMean, cg->myModeMeanSetting.minMean, cg->myModeMeanSetting.maxMean, "Nighttime Mean Target", false)) ok = false; - if (! validateFloat(&cg->myModeMeanSetting.mean_threshold, 0.0, 1.0, "Mean Threshold", false)) + if (! validateFloat(&cg->myModeMeanSetting.nightMean_threshold, cg->myModeMeanSetting.minMean_threshold, cg->myModeMeanSetting.maxMean_threshold, "Mean Threshold", false)) + ok = false; + if (! validateFloat(&cg->myModeMeanSetting.mean_p0, cg->myModeMeanSetting.minMean_p, cg->myModeMeanSetting.maxMean_p, "Mean p0", false)) + ok = false; + if (! validateFloat(&cg->myModeMeanSetting.mean_p0, cg->myModeMeanSetting.minMean_p, cg->myModeMeanSetting.maxMean_p, "Mean p1", false)) + ok = false; + if (! validateFloat(&cg->myModeMeanSetting.mean_p2, cg->myModeMeanSetting.minMean_p, cg->myModeMeanSetting.maxMean_p, "Mean p2", false)) ok = false; // If there's too short of a delay, pictures won't upload fast enough. @@ -1335,7 +1502,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateLong(&cg->flip, cc.MinValue, cc.MaxValue, "Flip", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_FLIP failed with %s\n", getRetCode(ret)); + Log(0, "%s: ASI_FLIP failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1347,13 +1514,13 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) { if (cg->rotation != 180) { - Log(0, "%s*** ERROR: Only 0 and 180 degrees are supported for rotation; you entered %ld.%s\n", c(KRED), cg->rotation, c(KNRM)); + Log(0, "*** %s: ERROR: Only 0 and 180 degrees are supported for rotation; you entered %ld.\n", cg->ME, cg->rotation); ok = false; } } else if (cg->rotation != 90 && cg->rotation != 180 && cg->rotation != 270) { - Log(0, "%s*** ERROR: Only 0, 90, 180, and 270 degrees are supported for rotation; you entered %ld.%s\n", c(KRED), cg->rotation, c(KNRM)); + Log(0, "*** %s: ERROR: Only 0, 90, 180, and 270 degrees are supported for rotation; you entered %ld.\n", cg->ME, cg->rotation); ok = false; } } @@ -1388,10 +1555,10 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) validateLong(&cg->overlay.linenumber, 0, sizeof(cg->overlay.linetype)-1, "Font Smoothness", true); if (cg->overlay.fc != NULL && sscanf(cg->overlay.fc, "%d %d %d", &cg->overlay.fontcolor[0], &cg->overlay.fontcolor[1], &cg->overlay.fontcolor[2]) != 3) - Log(-1, "%s*** WARNING: Not enough font color parameters: '%s'%s\n", c(KRED), cg->overlay.fc, c(KNRM)); + Log(-1, "*** %s: WARNING: Not enough font color parameters: '%s'\n", cg->ME, cg->overlay.fc); if (cg->overlay.sfc != NULL && sscanf(cg->overlay.sfc, "%d %d %d", &cg->overlay.smallFontcolor[0], &cg->overlay.smallFontcolor[1], &cg->overlay.smallFontcolor[2]) != 3) - Log(-1, "%s*** WARNING: Not enough small font color parameters: '%s'%s\n", c(KRED), cg->overlay.sfc, c(KNRM)); + Log(-1, "*** %s: WARNING: Not enough small font color parameters: '%s'%s\n", cg->ME, cg->overlay.sfc); cg->defaultBin = 1; if (! checkBin(cg->dayBin, ci, "Daytime Binning")) @@ -1407,18 +1574,10 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) ret = getControlCapForControlType(cg->cameraNumber, ASI_AUTO_TARGET_BRIGHTNESS, &cc); if (ret == ASI_SUCCESS) { - cg->defaultBrightness = cc.DefaultValue; // used elsewhere - if (cg->dayBrightness == NOT_CHANGED) - cg->dayBrightness = cc.DefaultValue; - else - validateLong(&cg->dayBrightness, cc.MinValue, cc.MaxValue, "Daytime Brightness", true); - - if (cg->nightBrightness == NOT_CHANGED) - cg->nightBrightness = cc.DefaultValue; - else - validateLong(&cg->nightBrightness, cc.MinValue, cc.MaxValue, "Nighttime Brightness", true); + validateLong(&cg->dayBrightness, cc.MinValue, cc.MaxValue, "Daytime Brightness", true); + validateLong(&cg->nightBrightness, cc.MinValue, cc.MaxValue, "Nighttime Brightness", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_AUTO_TARGET_BRIGHTNESS failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: ASI_AUTO_TARGET_BRIGHTNESS failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1436,7 +1595,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->nightGain, cc.MinValue, cc.MaxValue, "Nighttime Gain", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_GAIN failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: ASI_GAIN failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } ret = getControlCapForControlType(cg->cameraNumber, ASI_AUTO_MAX_GAIN, &cc); @@ -1452,7 +1611,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->nightMaxAutoGain, cc.MinValue, cc.MaxValue, "Nighttime Max Auto-Gain", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_AUTO_MAX_GAIN failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: ASI_AUTO_MAX_GAIN failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1471,7 +1630,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->nightWBR, cc.MinValue, cc.MaxValue, "Nighttime Red Balance", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_WB_R failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: ASI_WB_R failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } ret = getControlCapForControlType(cg->cameraNumber, ASI_WB_B, &cc); @@ -1488,7 +1647,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->nightWBB, cc.MinValue, cc.MaxValue, "Nighttime Blue Balance", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_WB_B failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: ASI_WB_B failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } } @@ -1503,7 +1662,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->saturation, cc.MinValue, cc.MaxValue, "Saturation", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "SATURATION failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: SATURATION failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1516,7 +1675,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->contrast, cc.MinValue, cc.MaxValue, "Contrast", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "CONTRAST failed with %s\n", getRetCode(ret)); + Log(0, "*** %s ERROR: CONTRAST failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1529,7 +1688,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateFloat(&cg->sharpness, cc.MinValue, cc.MaxValue, "Sharpness", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "SHARPNESS failed with %s\n", getRetCode(ret)); + Log(0, "*** %s ERROR: SHARPNESS failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } } @@ -1542,7 +1701,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateLong(&cg->gamma, cc.MinValue, cc.MaxValue, "gamma", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_GAMMA failed with %s\n", getRetCode(ret)); + Log(0, "*** %s ERROR: ASI_GAMMA failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1554,7 +1713,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateLong(&cg->offset, cc.MinValue, cc.MaxValue, "offset", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_OFFSET failed with %s\n", getRetCode(ret)); + Log(0, "*** %s ERROR: ASI_OFFSET failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } @@ -1572,7 +1731,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateLong(&cg->nightTargetTemp, cc.MinValue, cc.MaxValue, "Nighttime Target Sensor Temperature", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_TARGET_TEMP failed with %s\n", getRetCode(ret)); + Log(0, "*** %s ERROR: ASI_TARGET_TEMP failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } } @@ -1585,7 +1744,7 @@ bool validateSettings(config *cg, ASI_CAMERA_INFO ci) else validateLong(&cg->asiBandwidth, cc.MinValue, cc.MaxValue, "USB Bandwidth", true); } else if (ret != ASI_ERROR_INVALID_CONTROL_TYPE) { - Log(0, "ASI_BANDWIDTHOVERLOAD failed with %s\n", getRetCode(ret)); + Log(0, "*** %s: ERROR: ASI_BANDWIDTHOVERLOAD failed with %s\n", cg->ME, getRetCode(ret)); ok = false; } } diff --git a/src/Makefile b/src/Makefile index b766730c8..af7943044 100644 --- a/src/Makefile +++ b/src/Makefile @@ -1,6 +1,6 @@ arch = $(shell dpkg --print-architecture) platform = $(shell uname -m) -prefix = +prefix = sysconfdir = ${prefix}/etc exec_prefix = /usr @@ -42,29 +42,31 @@ endif DEFS = -D_LIN -D_DEBUG -DGLIBC_20 CFLAGS = -Werror -Wall -Wno-psabi -Wno-unused-result -g -O2 -lpthread -pthread +# Starting with the Feb 2023 Bullseye release, both 32- and 64-bit return +# $(platform) of "aarch64" on 64-bit hardware so we need to check $(arch), +# which returns "armhf" on 32-bit OS and "armhf" on 64-bit OS. +# Ditto i386 arch vs x86_64 platform. + ifeq ($(platform), armv6l) CC = arm-linux-gnueabihf-g++ AR= arm-linux-gnueabihf-ar CFLAGS += -march=armv6 CFLAGS += -lrt ZWOSDK = -Llib/armv6 -I./include -else ifeq ($(platform), x86_64) +else ifeq ($(arch), i386) CC = g++ AR= ar - ZWOSDK = -Llib/x64 -I./include -else ifeq ($(platform), i386) + ZWOSDK = -Llib/x86 -I./include +else ifeq ($(platform), x86_64) CC = g++ AR= ar - ZWOSDK = -Llib/x86 -I./include + ZWOSDK = -Llib/x64 -I./include else ifeq ($(arch), arm64) # Ubuntu 20.04 added by Jos Wennmacker. CC = g++ AR= ar ZWOSDK = -Llib/armv8 -I./include else # $(platform) = armv7l or $(arch) = armhf - # Starting with the Feb 2023 Bullseye release, both 32- and 64-bit return - # $(platform) of "aarch64" on 64-bit hardware so we need to check $(arch), - # which returns "armhf" on 32-bit OS and "armhf" on 64-bit OS. CC = arm-linux-gnueabihf-g++ AR= arm-linux-gnueabihf-ar CFLAGS += -march=armv7 -mthumb @@ -104,16 +106,10 @@ ifeq (,$(OPENCV)) endif .PHONY : check_deps -patchsunwait: - @echo `date +%F\ %R:%S` Initializing sunwait submodule... - @git submodule init - @git submodule update - @echo `date +%F\ %R:%s` Patching sunwait compile warnings... - @patch -p1 -d sunwait-src/ < sunwait.patch - @touch patchsunwait - -sunwait: patchsunwait +sunwait: @echo `date +%F\ %R:%S` Building sunwait... + @git submodule init + @git submodule update @$(MAKE) -C sunwait-src @cp sunwait-src/sunwait . @echo `date +%F\ %R:%S` Done. @@ -122,11 +118,11 @@ allsky_common.o: allsky_common.cpp include/allsky_common.h @echo Building $@ ... @$(CC) -c allsky_common.cpp -o $@ $(CFLAGS) $(OPENCV) -mode_mean.o: mode_mean.cpp include/mode_mean.h include/raspistill.h include/allsky_common.h +mode_mean.o: mode_mean.cpp include/mode_mean.h include/allsky_common.h @echo Building $@ ... @$(CC) -c mode_mean.cpp -o $@ $(CFLAGS) $(OPENCV) -capture_RPi.o: capture_RPi.cpp ASI_functions.cpp include/raspistill.h include/allsky_common.h +capture_RPi.o: capture_RPi.cpp ASI_functions.cpp include/mode_mean.h include/allsky_common.h @echo Building $@ ... @$(CC) -c capture_RPi.cpp -o $@ $(CFLAGS) $(OPENCV) @@ -211,4 +207,3 @@ endif # Correct directory structure check %: @echo `date +%F\ %R:%S` nothing to do for $@ - diff --git a/src/allsky_common.cpp b/src/allsky_common.cpp index 4d5861ca7..b48bdb589 100644 --- a/src/allsky_common.cpp +++ b/src/allsky_common.cpp @@ -40,7 +40,7 @@ static char const *fontnames[] = { // Character representation of names for cl void Log(int required_level, const char *fmt, ...) { if ((int)abs(CG.debugLevel) >= required_level) { - char msg[512]; + char msg[1000]; va_list va; va_start(va, fmt); vsnprintf(msg, sizeof(msg)-1, fmt, va); @@ -62,8 +62,9 @@ void Log(int required_level, const char *fmt, ...) *p = '\0'; } - char command[1024]; - snprintf(command, sizeof(command)-1, "%s/scripts/addMessage.sh %s '%s'", CG.allskyHome, severity, msg); + char command[sizeof(msg) + 100]; + snprintf(command, sizeof(command)-1, "%s/scripts/addMessage.sh %s '%s'", + CG.allskyHome, severity, msg); Log(4, "Executing %s\n", command); (void) system(command); } @@ -256,6 +257,12 @@ void add_variables_to_command(config cg, char *cmd, timeval startDateTime) snprintf(tmp, s, " MEAN=%s", LorF(cg.lastMean, "%d", "%f")); strcat(cmd, tmp); } + // FULLMEAN is to see if the mean of the whole image is the same as the mean returned + // by removeBadImages.sh; if so, removeBadImages.sh doesn't need to determine the mean. + if (cg.lastMeanFull >= 0.0) { + snprintf(tmp, s, " FULLMEAN=%s", LorF(cg.lastMeanFull, "%d", "%f")); + strcat(cmd, tmp); + } // Since negative temperatures are valid, check against an impossible temperature. // The temperature passed to us is 10 times the actual temperature so we can deal with @@ -306,54 +313,64 @@ void add_variables_to_command(config cg, char *cmd, timeval startDateTime) // Display a length of time in different units, depending on the length's value. // If the "multi" flag is set, display in multiple units if appropriate. +// Rotate buffers so we can call length_in_units() up to l_count times in one printf(). + +const int l_count = 3; // 3 buffers +const int l = 50; // each 50 long +int on_l = l_count; // point to last one +static char length[l_count][l+1]; +static char *length_p = length[l_count - 1]; // last one + char *length_in_units(long us, bool multi) // microseconds { - const int l = 50; - static char length[l]; + // Rotate buffers. + if (++on_l > l_count) + on_l = 1; + length_p = length[on_l - 1]; + if (us == 0) { - snprintf(length, l, "0 us"); + snprintf(length_p, l, "0 us"); + return(length_p); } - else + + double us_in_ms = (double)us / US_IN_MS; + double abs_us_in_ms = abs(us_in_ms); + // The boundaries on when to display one or two units are really a matter of taste. + if (abs_us_in_ms < 0.5) // less than 0.5 ms { - double us_in_ms = (double)us / US_IN_MS; - double abs_us_in_ms = abs(us_in_ms); - // The boundaries on when to display one or two units are really a matter of taste. - if (abs_us_in_ms < 0.5) // less than 0.5 ms - { - snprintf(length, l, "%'ld us", us); - } - else if (abs_us_in_ms < 1.5) // between 0.5 ms and 1.5 ms - { - if (multi) - snprintf(length, l, "%'ld us (%.3f ms)", us, us_in_ms); - else - snprintf(length, l, "%'ld us", us); - } - else if (abs_us_in_ms < (0.5 * MS_IN_SEC)) // 1.5 ms to 0.5 sec - { - // Don't display seconds if it'll look like (0.00 sec). - double s = (double)us / US_IN_SEC; - if (multi && s >= 0.005) { - snprintf(length, l, "%'.2f ms (%.2lf sec)", us_in_ms, s); - } else { - snprintf(length, l, "%'.2f ms", us_in_ms); - } - } - else if (abs_us_in_ms < (1.0 * MS_IN_SEC)) // between 0.5 sec and 1 sec - { - if (multi) - snprintf(length, l, "%'.2f ms (%.2lf sec)", us_in_ms, (double)us / US_IN_SEC); - else - snprintf(length, l, "%'.1f ms", us_in_ms); - } - else // over 1 sec - { - snprintf(length, l, "%'.1lf sec", (double)us / US_IN_SEC); + snprintf(length_p, l, "%'ld us", us); + } + else if (abs_us_in_ms < 1.5) // between 0.5 ms and 1.5 ms + { + if (multi) + snprintf(length_p, l, "%'ld us (%.3f ms)", us, us_in_ms); + else + snprintf(length_p, l, "%'ld us", us); + } + else if (abs_us_in_ms < (0.5 * MS_IN_SEC)) // 1.5 ms to 0.5 sec + { + // Don't display seconds if it'll look like (0.00 sec). + double s = (double)us / US_IN_SEC; + if (multi && s >= 0.005) { + snprintf(length_p, l, "%'.2f ms (%.2lf sec)", us_in_ms, s); + } else { + snprintf(length_p, l, "%'.2f ms", us_in_ms); } - } - return(length); + else if (abs_us_in_ms < (1.0 * MS_IN_SEC)) // between 0.5 sec and 1 sec + { + if (multi) + snprintf(length_p, l, "%'.2f ms (%.2lf sec)", us_in_ms, (double)us / US_IN_SEC); + else + snprintf(length_p, l, "%'.1f ms", us_in_ms); + } + else // over 1 sec + { + snprintf(length_p, l, "%'.1lf sec", (double)us / US_IN_SEC); + } + + return(length_p); } // sunwait barfs if it receives an angle with a comma in it, @@ -380,7 +397,9 @@ std::string calculateDayOrNight(const char *latitude, const char *longitude, flo char sunwaitCommand[128]; int d; - sprintf(sunwaitCommand, "sunwait poll exit angle %s %s %s > /dev/null", convertCommaToPeriod(angle, "%.4f"), latitude, longitude); + snprintf(sunwaitCommand, sizeof(sunwaitCommand), + "sunwait poll exit angle %s %s %s > /dev/null", + convertCommaToPeriod(angle, "%.4f"), latitude, longitude); Log(4, "Executing %s\n", sunwaitCommand); d = system(sunwaitCommand); // returns exit code 2 for DAY, 3 for night @@ -389,14 +408,14 @@ std::string calculateDayOrNight(const char *latitude, const char *longitude, flo d = WEXITSTATUS(d); if (d != 2 && d != 3) { - Log(0, "*** ERROR: sunwait returned %d, not DAY or NIGHT\n", d); + Log(0, "*** %s: ERROR: sunwait returned %d, not DAY or NIGHT\n", CG.ME, d); closeUp(EXIT_ERROR_STOP); } return(d == 2 ? _day : _night); } // Didn't exit normally - Log(0, "*** ERROR: sunwait exited abnormally: 0x%x\n", d); + Log(0, "*** %s: ERROR: sunwait exited abnormally: 0x%x\n", CG.ME, d); closeUp(EXIT_ERROR_STOP); /*NOTREACHED*/ return(""); @@ -407,7 +426,9 @@ int calculateTimeToNightTime(const char *latitude, const char *longitude, float { std::string t; char sunwaitCommand[128]; // returns "hh:mm" - sprintf(sunwaitCommand, "sunwait list set angle %s %s %s", convertCommaToPeriod(angle, "%.4f"), latitude, longitude); + snprintf(sunwaitCommand, sizeof(sunwaitCommand), + "sunwait list set angle %s %s %s", + convertCommaToPeriod(angle, "%.4f"), latitude, longitude); t = exec(sunwaitCommand); t.erase(std::remove(t.begin(), t.end(), '\n'), t.end()); @@ -417,7 +438,8 @@ int calculateTimeToNightTime(const char *latitude, const char *longitude, float // after midnight or before noon. if (sscanf(t.c_str(), "%d:%d", &hNight, &mNight) != 2) { - Log(0, "ERROR: With angle %.4f sunwait returned unknown time to nighttime: %s\n", angle, t.c_str()); + Log(0, "*** %s: ERROR: With angle %.4f sunwait returned unknown time to nighttime: %s\n", + CG.ME, angle, t.c_str()); return(1 * S_IN_HOUR); // 1 hour - should we exit instead? } secsNight = (hNight * S_IN_HOUR) + (mNight * S_IN_MIN); // secs to nighttime from start of today @@ -465,7 +487,7 @@ bool getBoolean(const char* arg) if (strcasecmp(arg, "no") != 0 && strcasecmp(arg, "false") != 0 && strcasecmp(arg, "0") != 0) { - Log(0, "*** WARNING: argument '%s' is not a boolean; setting to 'false'\n", arg); + Log(0, "*** %s: WARNING: argument '%s' is not a boolean; setting to 'false'\n", CG.ME, arg); } return(false); } @@ -476,14 +498,14 @@ bool checkForValidExtension(config *cg) { char const *ext = strrchr(cg->fileName, '.'); // e.g., "image.jpg" if (ext == NULL) { - Log(0, "*** ERROR: No extension given on filename: [%s]\n", cg->fileName); + Log(0, "*** %s: ERROR: No extension given on filename: [%s]\n", CG.ME, cg->fileName); return(false); } ext++; if (strcasecmp(ext, "jpg") == 0 || strcasecmp(ext, "jpeg") == 0) { if (cg->imageType == IMG_RAW16) { - Log(0, "*** ERROR: RAW16 images only work with .png files; either change the Image Type or the Filename.\n"); + Log(0, "*** %s: ERROR: RAW16 images only work with .png files; either change the Image Type or the Filename.\n", cg->ME); return(false); } cg->imageExt = "jpg"; @@ -515,7 +537,7 @@ bool checkForValidExtension(config *cg) } } else { - Log(0, "*** ERROR: Unsupported image extension (%s); only .jpg and .png are supported.\n", ext); + Log(0, "*** %s: ERROR: Unsupported image extension (%s); only .jpg and .png are supported.\n", cg->ME, ext); return(false); } @@ -674,9 +696,9 @@ int doOverlay(cv::Mat image, config cg, char *startTime, int gainChange) // Display these messages every time, since it's possible the user will // correct the issue while we're running. if (access(cg.overlay.ImgExtraText, F_OK ) == -1 ) { - Log(1, " > *** WARNING: Extra Text File Does Not Exist So Ignoring It\n"); + Log(1, " > *** %s: WARNING: Extra Text File Does Not Exist So Ignoring It\n", cg.ME); } else if (access(cg.overlay.ImgExtraText, R_OK ) == -1 ) { - Log(1, " > *** WARNING: Cannot Read From Extra Text File So Ignoring It\n"); + Log(1, " > *** %s: WARNING: Cannot Read From Extra Text File So Ignoring It\n", cg.ME); } else { FILE *fp = fopen(cg.overlay.ImgExtraText, "r"); @@ -696,7 +718,7 @@ int doOverlay(cv::Mat image, config cg, char *startTime, int gainChange) Log(4, ", so Ignoring\n"); } } else { - Log(0, " > *** ERROR: Stat Of Extra Text File Failed !\n"); + Log(0, " > *** %s: ERROR: Stat Of Extra Text File Failed !\n", cg.ME); } } else { bAddExtra = true; @@ -723,7 +745,7 @@ int doOverlay(cv::Mat image, config cg, char *startTime, int gainChange) } fclose(fp); } else { - Log(1, " > *** WARNING: Failed To Open Extra Text File\n"); + Log(1, " > *** %s: WARNING: Failed To Open Extra Text File\n", cg.ME); } } } @@ -835,19 +857,19 @@ void IntHandle(int i) gotSignal = false; // TODO: Re-read configuration instead of restarting. - Log(4, "Got SIGHUP to restart.\n"); + Log(4, "%s: Got SIGHUP to restart.\n", CG.ME); closeUp(EXIT_RESTARTING); /*NOTREACHED*/ } if (i == SIGINT || i == SIGTERM) { - Log(4, "Got %s to exit.\n", i == SIGINT ? "SIGINT" : "SIGTERM"); + Log(4, "%s: Got %s to exit.\n", CG.ME, i == SIGINT ? "SIGINT" : "SIGTERM"); closeUp(EXIT_OK); } else { - Log(0, "Got unknown signal %d.\n", i); + Log(0, "%s: Got unknown signal %d.\n", CG.ME, i); closeUp(i); } } @@ -970,6 +992,7 @@ void displayHelp(config cg) printf(" -%-*s - Maximum daytime auto-exposure in ms.\n", n, "daymaxexposure n"); printf(" -%-*s - Daytime exposure in us [%'ld].\n", n, "dayexposure n", cg.dayExposure_us); printf(" -%-*s - Daytime mean target brightness [%.2f].\n", n, "daymean", cg.myModeMeanSetting.dayMean); + printf(" -%-*s - Daytime mean target threshold [%.2f].\n", n, "daymean-threshold n", cg.myModeMeanSetting.dayMean_threshold); printf(" %-*s NOTE: Daytime auto-gain and auto-exposure should be on for best results.\n", n, ""); printf(" -%-*s - Daytime brightness change [%'ld].\n", n, "daybrightness n", cg.dayBrightness); printf(" -%-*s - Delay between daytime images in ms [%'ld].\n", n, "dayDelay n", cg.dayDelay_ms); @@ -992,9 +1015,10 @@ void displayHelp(config cg) printf("\nNighttime settings:\n"); printf(" -%-*s - 1 enables nighttime auto-exposure [%s].\n", n, "nightautoexposure b", yesNo(cg.nightAutoExposure)); printf(" -%-*s - Maximum nighttime auto-exposure in ms.\n", n, "nightmaxexposure n"); - printf(" -%-*s - Nighttime exposure in us [%'ld].\n", n, "nightexposure n", cg.nightExposure_us * US_IN_MS); + printf(" -%-*s - Nighttime exposure in us [%'ld].\n", n, "nightexposure n", cg.nightExposure_us); printf(" -%-*s - Nighttime mean target brightness [%.2f].\n", n, "nightmean n", cg.myModeMeanSetting.nightMean); printf(" %-*s NOTE: Nighttime auto-gain and auto-exposure should be on for best results.\n", n, ""); + printf(" -%-*s - Nighttime mean target threshold [%.2f].\n", n, "nightmean-threshold n", cg.myModeMeanSetting.nightMean_threshold); printf(" -%-*s - Nighttime brightness change [%ld].\n", n, "nightbrightness n n", cg.nightBrightness); printf(" -%-*s - Delay between nighttime images in ms [%'ld].\n", n, "nightDelay n", cg.nightDelay_ms); printf(" -%-*s - 1 enables nighttime auto gain [%s].\n", n, "nightautogain b", yesNo(cg.nightAutoGain)); @@ -1027,7 +1051,7 @@ void displayHelp(config cg) } printf(" -%-*s - Camera maximum width [%ld].\n", n, "width n", cg.width); printf(" -%-*s - Camera maximum height [%ld].\n", n, "height n", cg.height); - printf(" -%-*s - Type of image: 99 = auto, 0 = RAW8, 1 = RGB24 [%ld].\n", n, "type n", cg.imageType); + printf(" -%-*s - Type of image: 99 = auto, 0 = RAW8, 1 = RGB24 [%ld]", n, "type n", cg.imageType); if (cg.ct == ctZWO) { printf(", 2 = RAW16, 3 = Y8"); } @@ -1042,19 +1066,21 @@ void displayHelp(config cg) } printf(" -%-*s - 0 = No flip, 1 = Horizontal, 2 = Vertical, 3 = Both [%ld].\n", n, "flip n", cg.flip); printf(" -%-*s - 1 enables consistent delays between images [%s].\n", n, "consistentDelays b", yesNo(cg.consistentDelays)); + printf(" -%-*s - Format the time is displayed in [%s].\n", n, "timeformat s", cg.timeFormat); printf(" -%-*s - 1 enables notification images, for example, 'Camera is off during day' [%s].\n", n, "notificationimages b", yesNo(cg.notificationImages)); printf(" -%-*s - Latitude of the camera [no default - you must set it].\n", n, "latitude s"); printf(" -%-*s - Longitude of the camera [no default - you must set it].\n", n, "longitude s"); printf(" -%-*s - Angle of the sun below the horizon [%.2f].\n", n, "angle n", cg.angle); printf(" %-*s -6 = civil twilight -12 = nautical twilight -18 = astronomical twilight.\n", n, ""); printf(" -%-*s - 1 enables capturing of daytime images [%s].\n", n, "takeDaytimeImages b", yesNo(cg.daytimeCapture)); - printf(" -%-*s - 1 takes dark frames and disables the overlay [%s].\n", n, "takeDarkFrames b", yesNo(cg.takeDarkFrames)); + printf(" -%-*s - 1 takes dark frames [%s].\n", n, "takeDarkFrames b", yesNo(cg.takeDarkFrames)); printf(" -%-*s - Your locale - to determine thousands separator and decimal point [%s].\n", n, "locale s", "locale on Pi"); printf(" %-*s Type 'locale' at a command prompt to determine yours.\n", n, ""); if (cg.ct == ctZWO) { printf(" -%-*s - Default = %d %d %0.2f %0.2f (box width X, box width y, X offset percent (0-100), Y offset (0-100))\n", n, "histogrambox n n n n", cg.HB.histogramBoxSizeX, cg.HB.histogramBoxSizeY, cg.HB.histogramBoxPercentFromLeft * 100.0, cg.HB.histogramBoxPercentFromTop * 100.0); printf(" -%-*s - 1 enables auto USB Speed.\n", n, "autousb b"); printf(" -%-*s - USB bandwidth percent.\n", n, "usb n"); + printf(" -%-*s - 1 enables a newer ZWO auto-exposure algorithm [%s].\n", n, "experimentalExposure b", yesNo(cg.HB.useExperimentalExposure)); printf(" -%-*s - Determines if version 0.8 exposure method should be used [%s].\n", n, "newexposure b", yesNo(cg.videoOffBetweenImages)); } if (cg.ct == ctRPi) { @@ -1065,12 +1091,9 @@ void displayHelp(config cg) printf("\nOverlay settings:\n"); printf(" -%-*s - Set to %d to use the new, enhanced 'module' overlay program [%s].\n", n, "overlayMethod n", OVERLAY_METHOD_LEGACY, getOverlayMethod(cg.overlay.overlayMethod).c_str()); printf(" -%-*s - Set to 1 to display the time [%s].\n", n, "showTime b", yesNo(cg.overlay.showTime)); - printf(" -%-*s - Format the optional time is displayed in [%s].\n", n, "timeformat s", cg.timeFormat); - printf(" -%-*s - 1 displays the exposure length [%s].\n", n, "showExposure b", yesNo(cg.overlay.showExposure)); - if (cg.ct == ctZWO) { - printf(" -%-*s - 1 displays the camera sensor temperature [%s].\n", n, "showTemp b", yesNo(cg.overlay.showTemp)); - } printf(" -%-*s - Units to display temperature in: 'C'elsius, 'F'ahrenheit, or 'B'oth [%s].\n", n, "temptype s", cg.tempType); + printf(" -%-*s - 1 displays the exposure length [%s].\n", n, "showExposure b", yesNo(cg.overlay.showExposure)); + printf(" -%-*s - 1 displays the camera sensor temperature [%s].\n", n, "showTemp b", yesNo(cg.overlay.showTemp)); printf(" -%-*s - 1 displays the gain [%s].\n", n, "showGain b", yesNo(cg.overlay.showGain)); printf(" -%-*s - 1 displays the brightness [%s].\n", n, "showBrightness b", yesNo(cg.overlay.showBrightness)); printf(" -%-*s - 1 displays the mean brightness used in auto-exposure [%s].\n", n, "showMean b", yesNo(cg.overlay.showMean)); @@ -1098,12 +1121,11 @@ void displayHelp(config cg) printf(" -%-*s - Where to save 'filename' [%s].\n", n, "save_dir s", cg.saveDir); printf(" -%-*s - 1 previews the captured images. Only works with a Desktop Environment [%s]\n", n, "preview", yesNo(cg.preview)); printf(" -%-*s - Outputs the camera's capabilities to the specified file and exists.\n", n, "cc_file s"); - printf(" -%-*s - Set mean-value and activates exposure control [%.2f].\n", n, "mean-threshold n", cg.myModeMeanSetting.mean_threshold); if (cg.ct == ctRPi) { printf(" -%-*s - Command being used to take pictures (Buster: raspistill, Bullseye: libcamera-still\n", n, "cmd s"); } /* These are too advanced for anyone other than developers. - printf(" -%-*s - Be careful changing these values, ExposureChange (Steps) = p0 + (p1*diff) + (p2*diff)^2 [%.1f].\n", n, "mean-p0 n", cg.myModeMeanSetting.mean_threshold); + printf(" -%-*s - Be careful changing these values, ExposureChange (Steps) = p0 + (p1*diff) + (p2*diff)^2 [%.1f].\n", n, "mean-p0 n", cg.myModeMeanSetting.dayMean_threshold); printf(" -%-*s - [%.1f].\n", n, "mean-p1 n", cg.myModeMeanSetting.mean_p1); printf(" -%-*s - [%.1f].\n", n, "mean-p2 n", cg.myModeMeanSetting.mean_p2); */ @@ -1173,20 +1195,20 @@ void displaySettings(config cg) if (cg.nightAutoGain) printf(", Max Auto-Gain: %s", LorF(cg.nightMaxAutoGain, "%ld", "%1.2f")); printf("\n"); - if (cg.supportsMyModeMean) - printf(" Mean Value (day): %1.3f\n", cg.myModeMeanSetting.dayMean); - if (cg.supportsMyModeMean) - printf(" Mean Value (night): %1.3f\n", cg.myModeMeanSetting.nightMean); + if (cg.gainTransitionTimeImplemented) + printf(" Gain Transition Time: %.1f minutes\n", (float) cg.gainTransitionTime/60); + + printf(" Target Mean Value (day): %1.3f\n", cg.myModeMeanSetting.dayMean); + printf(" Target Mean Value (night): %1.3f\n", cg.myModeMeanSetting.nightMean); + printf(" Target Mean Threshold (day): %1.3f\n", cg.myModeMeanSetting.dayMean_threshold); + printf(" Target Mean Threshold (night): %1.3f\n", cg.myModeMeanSetting.nightMean_threshold); if (cg.supportsMyModeMean) { - printf(" Threshold: %1.3f:\n", cg.myModeMeanSetting.mean_threshold); printf(" p0: %1.3f\n", cg.myModeMeanSetting.mean_p0); printf(" p1: %1.3f\n", cg.myModeMeanSetting.mean_p1); printf(" p2: %1.3f\n", cg.myModeMeanSetting.mean_p2); } - if (cg.gainTransitionTimeImplemented) - printf(" Gain Transition Time: %.1f minutes\n", (float) cg.gainTransitionTime/60); printf(" Brightness (day): %ld\n", cg.dayBrightness); printf(" Brightness (night): %ld\n", cg.nightBrightness); printf(" Binning (day): %ld\n", cg.dayBin); @@ -1235,6 +1257,7 @@ void displaySettings(config cg) cg.HB.histogramBoxSizeX, cg.HB.histogramBoxSizeY, cg.HB.histogramBoxPercentFromLeft * 100.0, cg.HB.histogramBoxPercentFromTop * 100.0, cg.HB.centerX, cg.HB.centerY, cg.HB.leftOfBox, cg.HB.topOfBox, cg.HB.rightOfBox, cg.HB.bottomOfBox); + printf(" New Exposure Algorithm: %s\n", yesNo(cg.HB.useExperimentalExposure)); printf(" Video OFF Between Images: %s\n", yesNo(cg.videoOffBetweenImages)); } printf(" Preview: %s\n", yesNo(cg.preview)); @@ -1273,7 +1296,7 @@ void displaySettings(config cg) printf(" Show Histogram Box: %s\n", yesNo(cg.overlay.showHistogramBox)); } } else if (cg.supportsTemperature) { - printf(" Temperature type: %s\n", stringORnone(cg.tempType)); + printf(" Temperature type: %s\n", stringORnone(cg.tempType)); } printf(" Allsky version: %s\n", stringORnone(cg.version)); @@ -1392,12 +1415,12 @@ static bool getConfigFileArguments(config *cg) { if (called_from_getConfigFileArguments) { - Log(-1, "*** WARNING: Configuration file calls itself; ignoring!\n"); + Log(-1, "*** %s: WARNING: Configuration file calls itself; ignoring!\n", cg->ME); return true; } if (cg->configFile[0] == '\0') { - Log(0, "*** ERROR: Unable to read configuration file: no file specified!\n"); + Log(0, "*** %s: ERROR: Unable to read configuration file: no file specified!\n", cg->ME); return false; } @@ -1407,27 +1430,31 @@ static bool getConfigFileArguments(config *cg) if ((fd = open(cg->configFile, O_RDONLY)) == -1) { int e = errno; - Log(0, "*** ERROR: Could not open configuration file '%s': %s!", cg->configFile, strerror(e)); + Log(0, "*** %s: ERROR: Could not open configuration file '%s': %s!", + cg->ME, cg->configFile, strerror(e)); return false; } struct stat statbuf; if (fstat(fd, &statbuf) == 1) // This should never fail { int e = errno; - Log(0, "*** ERROR: Could not fstat() configuration file '%s': %s!", cg->configFile, strerror(e)); + Log(0, "*** %s: ERROR: Could not fstat() configuration file '%s': %s!", + cg->ME, cg->configFile, strerror(e)); return false; } // + 1 for trailing NULL if ((buf = (char *) realloc(buf, statbuf.st_size + 1)) == NULL) { int e = errno; - Log(0, "*** ERROR: Could not malloc() configuration file '%s': %s!", cg->configFile, strerror(e)); + Log(0, "*** %s: ERROR: Could not malloc() configuration file '%s': %s!", + cg->ME, cg->configFile, strerror(e)); return false; } if (read(fd, buf, statbuf.st_size) != statbuf.st_size) { int e = errno; - Log(0, "*** ERROR: Could not read() configuration file '%s': %s!", cg->configFile, strerror(e)); + Log(0, "*** %s: ERROR: Could not read() configuration file '%s': %s!", + cg->ME, cg->configFile, strerror(e)); return false; } buf[statbuf.st_size] = '\0'; @@ -1451,7 +1478,8 @@ static bool getConfigFileArguments(config *cg) if (*line == '=') // still at start of line { - Log(-1, "*** WARNING: Line %d in configuration file '%s' has nothing before '='!\n", lineNum, cg->configFile); + Log(-1, "*** %s: WARNING: Line %d in configuration file '%s' has nothing before '='!\n", + cg->ME, lineNum, cg->configFile); continue; } @@ -1473,7 +1501,8 @@ static bool getConfigFileArguments(config *cg) if (argc == 1) { - Log(-1, "*** WARNING: configuration file '%s' has no valid entries!\n", cg->configFile); + Log(-1, "*** %s: WARNING: configuration file '%s' has no valid entries!\n", + cg->ME, cg->configFile); } // Let's hope the config file doesn't call itself! @@ -1582,17 +1611,11 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) } else if (strcmp(a, "daymean") == 0) { -// TODO: this check should be done in capture program with all the other argument checks - // If the user specified 0.0, that means don't use modeMean auto exposure/gain. - double m = atof(argv[++i]); - if (m == 0.0) - { - cg->myModeMeanSetting.dayMean = 0.0; - } - else - { - cg->myModeMeanSetting.dayMean = std::min(1.0,std::max(0.0,m)); - } + cg->myModeMeanSetting.dayMean = atof(argv[++i]); + } + else if (strcmp(a, "daymeanthreshold") == 0) + { + cg->myModeMeanSetting.dayMean_threshold = atof(argv[++i]); } else if (strcmp(a, "daybrightness") == 0) { @@ -1662,16 +1685,11 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) } else if (strcmp(a, "nightmean") == 0) { -// TODO: this check should be done in capture program with all the other argument checks - double m = atof(argv[++i]); - if (m == 0.0) - { - cg->myModeMeanSetting.nightMean = 0.0; - } - else - { - cg->myModeMeanSetting.nightMean = std::min(1.0,std::max(0.0,m)); - } + cg->myModeMeanSetting.nightMean = atof(argv[++i]); + } + else if (strcmp(a, "nightmeanthreshold") == 0) + { + cg->myModeMeanSetting.nightMean_threshold = atof(argv[++i]); } else if (strcmp(a, "nightbrightness") == 0) { @@ -1771,23 +1789,21 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) { cg->userQuality = cg->quality = atol(argv[++i]); } - else if (strcmp(a, "meanthreshold") == 0) - { -// TODO: this check should be done in capture program with all the other argument checks - // Must be between 0.01 and 0.1. - cg->myModeMeanSetting.mean_threshold = std::min(0.1, std::max(0.01,atof(argv[++i]))); - } +else if (strcmp(a, "meanthreshold") == 0) // TODO: xxxxx delete in next release +{ + cg->myModeMeanSetting.mean_threshold = atof(argv[++i]); +} else if (strcmp(a, "meanp0") == 0) { - cg->myModeMeanSetting.mean_p0 = std::min(50.0, std::max(0.0,atof(argv[++i]))); + cg->myModeMeanSetting.mean_p0 = atof(argv[++i]); } else if (strcmp(a, "meanp1") == 0) { - cg->myModeMeanSetting.mean_p1 = std::min(50.0, std::max(0.0,atof(argv[++i]))); + cg->myModeMeanSetting.mean_p1 = atof(argv[++i]); } else if (strcmp(a, "meanp2") == 0) { - cg->myModeMeanSetting.mean_p2 = std::min(50.0, std::max(0.0,atof(argv[++i]))); + cg->myModeMeanSetting.mean_p2 = atof(argv[++i]); } else if (strcmp(a, "autousb") == 0) { @@ -1815,7 +1831,7 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) } else if (strcmp(a, "consistentdelays") == 0) { - cg->consistentDelays = getBoolean(argv[++i]); + cg->consistentDelays = getBoolean(argv[++i]); } else if (strcmp(a, "latitude") == 0) { @@ -1845,6 +1861,10 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) { cg->debugLevel = atol(argv[++i]); } + else if (strcmp(a, "experimentalexposure") == 0) + { + cg->HB.useExperimentalExposure = getBoolean(argv[++i]); + } else if (strcmp(a, "newexposure") == 0) { cg->videoOffBetweenImages = getBoolean(argv[++i]); @@ -1977,7 +1997,7 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) } else - Log(-1, "*** WARNING: Unknown argument: [%s]. Ignored.\n", a); + Log(-1, "*** %s: WARNING: Unknown argument: [%s]. Ignored.\n", cg->ME, a); } @@ -1987,7 +2007,8 @@ bool getCommandLineArguments(config *cg, int argc, char *argv[]) // If we are in "help" mode then we won't take picture AND won't produce CC info. if (cg->saveDir == NULL && cg->CC_saveFile == NULL && ! cg->help) { cg->saveDir = cg->allskyHome; - Log(-1, "*** WARNING: No directory to save images was specified. Using: [%s]\n", cg->saveDir); + Log(-1, "*** %s: WARNING: No directory to save images was specified. Using: [%s]\n", + cg->ME, cg->saveDir); } return(true); @@ -2004,7 +2025,7 @@ static char const *validateLatLong( char const *name) { if (l == NULL || *l == '\0') { - Log(0, "*** ERROR: %s not specified!\n", name); + Log(0, "*** %s: ERROR: %s not specified!\n", CG.ME, name); return(NULL); } @@ -2013,8 +2034,8 @@ static char const *validateLatLong( char direction = (char) toupper(l[len-1]); if (direction == positive || direction == negative) { if (l[0] == '+' || l[0] == '-') { - Log(0, "*** ERROR: %s cannot have BOTH + or - AND %c or %c. You entered [%s].\n", - name, positive, negative, l); + Log(0, "*** %s: ERROR: %s cannot have BOTH + or - AND %c or %c. You entered [%s].\n", + CG.ME, name, positive, negative, l); return(NULL); } return(l); @@ -2047,13 +2068,13 @@ void doLocale(config *cg) if (cg->locale == NULL) { cg->locale = setlocale(LC_NUMERIC, NULL); if (cg->locale == NULL) { - Log(-1, "*** WARNING: Could not get locale from Pi!\n"); + Log(-1, "*** %s: WARNING: Could not get locale from Pi!\n", cg->ME); } else { static char locale[100]; strncpy(locale, cg->locale, sizeof(locale)-1); cg->locale = locale; } } else if (setlocale(LC_NUMERIC, cg->locale) == NULL && ! cg->saveCC) { - Log(-1, "*** WARNING: Could not set locale to %s.\n", cg->locale); + Log(-1, "*** %s: WARNING: Could not set locale to %s.\n", cg->ME, cg->locale); } } diff --git a/src/capture_RPi.cpp b/src/capture_RPi.cpp index eb50f9280..b539d19fa 100644 --- a/src/capture_RPi.cpp +++ b/src/capture_RPi.cpp @@ -23,7 +23,6 @@ // When it's passed, functions call it "cg", so use upper case for global version. config CG; -#include "include/raspistill.h" #include "include/mode_mean.h" #define CAMERA_TYPE "RPi" @@ -358,13 +357,15 @@ int RPicapture(config cg, cv::Mat *image) { *image = cv::imread(cg.fullFilename, cv::IMREAD_UNCHANGED); if (! image->data) { - Log(1, "WARNING: Error re-reading file '%s'; skipping further processing.\n", basename(cg.fullFilename)); + Log(1, "*** %s: WARNING: Error re-reading file '%s'; skipping further processing.\n", + cg.ME, basename(cg.fullFilename)); } } } else if (! WIFSIGNALED(ret)) { - Log(1, " >>> WARNING: Unable to take picture, return code=0x%0x (%d)\n", ret, ret >> 8); + Log(1, " >>> %s: WARNING: Unable to take picture, return code=0x%0x (%d)\n", + cg.ME, ret, ret >> 8); Log(3, " Executed: %s\n", cmd); } // don't display message if we got a signal - that's done elsewhere. @@ -382,7 +383,7 @@ int main(int argc, char *argv[]) static char *a = getenv("ALLSKY_HOME"); // This must come before anything else if (a == NULL) { - Log(0, "%s: ERROR: ALLSKY_HOME not set!\n", CG.ME); + Log(0, "*** %s: ERROR: ALLSKY_HOME not set!\n", CG.ME); exit(EXIT_ERROR_STOP); } else @@ -436,16 +437,17 @@ int main(int argc, char *argv[]) processConnectedCameras(); // exits on error ASI_CAMERA_INFO ASICameraInfo; + // This gives a segmentation fault if cameraNumber isn't connected. asiRetCode = ASIGetCameraProperty(&ASICameraInfo, CG.cameraNumber); if (asiRetCode != ASI_SUCCESS) { - Log(0, "ERROR: ASIGetCamerProperty() returned: %s\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: ASIGetCamerProperty() returned: %s\n", CG.ME, getRetCode(asiRetCode)); exit(EXIT_ERROR_STOP); } asiRetCode = ASIGetNumOfControls(CG.cameraNumber, &iNumOfCtrl); if (asiRetCode != ASI_SUCCESS) { - Log(0, "ERROR: ASIGetNumOfControls() returned: %s\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: ASIGetNumOfControls() returned: %s\n", CG.ME, getRetCode(asiRetCode)); exit(EXIT_ERROR_STOP); } @@ -525,7 +527,7 @@ int main(int argc, char *argv[]) } else { - Log(0, "*** ERROR: Unknown Image Type: %d\n", CG.imageType); + Log(0, "*** %s: ERROR: Unknown Image Type: %d\n", CG.ME, CG.imageType); exit(EXIT_ERROR_STOP); } @@ -533,12 +535,10 @@ int main(int argc, char *argv[]) //------------------------------------------------------------------------------------------------------- // TODO: after merging myModeMeanSetting into CG.myModeMeanSetting, delete these lines. - myModeMeanSetting.dayMean = CG.myModeMeanSetting.dayMean; - myModeMeanSetting.nightMean = CG.myModeMeanSetting.nightMean; - myModeMeanSetting.mean_threshold = CG.myModeMeanSetting.mean_threshold; - myModeMeanSetting.mean_p0 = CG.myModeMeanSetting.mean_p0; - myModeMeanSetting.mean_p1 = CG.myModeMeanSetting.mean_p1; - myModeMeanSetting.mean_p2 = CG.myModeMeanSetting.mean_p2; +myModeMeanSetting.dayMean = CG.myModeMeanSetting.dayMean; +myModeMeanSetting.nightMean = CG.myModeMeanSetting.nightMean; +myModeMeanSetting.dayMean_threshold = CG.myModeMeanSetting.dayMean_threshold; +myModeMeanSetting.nightMean_threshold = CG.myModeMeanSetting.nightMean_threshold; displaySettings(CG); @@ -577,9 +577,11 @@ int main(int argc, char *argv[]) CG.currentDelay_ms = CG.nightDelay_ms; CG.currentBin = CG.nightBin; CG.currentGain = CG.nightGain; - CG.currentMaxAutoGain = CG.nightMaxAutoGain; // not needed since we're not using auto gain, but set to be consistent + // not needed since we're not using auto gain, but set to be consistent + CG.currentMaxAutoGain = CG.nightMaxAutoGain; CG.currentAutoGain = false; CG.myModeMeanSetting.currentMean = NOT_SET; + CG.myModeMeanSetting.currentMean_threshold = NOT_SET; if (CG.isCooledCamera) { CG.currentEnableCooler = CG.nightEnableCooler; @@ -615,36 +617,34 @@ int main(int argc, char *argv[]) continue; } - else + Log(1, "==========\n=== Starting daytime capture ===\n==========\n"); + + if (numExposures == 0 && CG.dayAutoExposure) + CG.currentSkipFrames = CG.daySkipFrames; + // We only skip initial frames if we are starting in daytime and using auto-exposure. + CG.currentAutoExposure = CG.dayAutoExposure; + CG.currentExposure_us = CG.dayExposure_us; + CG.currentMaxAutoExposure_us = CG.dayMaxAutoExposure_us; + CG.currentBrightness = CG.dayBrightness; + if (CG.isColorCamera) { - Log(1, "==========\n=== Starting daytime capture ===\n==========\n"); - - if (numExposures == 0 && CG.dayAutoExposure) - CG.currentSkipFrames = CG.daySkipFrames; - // We only skip initial frames if we are starting in daytime and using auto-exposure. - CG.currentAutoExposure = CG.dayAutoExposure; - CG.currentExposure_us = CG.dayExposure_us; - CG.currentMaxAutoExposure_us = CG.dayMaxAutoExposure_us; - CG.currentBrightness = CG.dayBrightness; - if (CG.isColorCamera) - { - CG.currentAutoAWB = CG.dayAutoAWB; - CG.currentWBR = CG.dayWBR; - CG.currentWBB = CG.dayWBB; - } - CG.currentDelay_ms = CG.dayDelay_ms; - CG.currentBin = CG.dayBin; - CG.currentGain = CG.dayGain; - CG.currentMaxAutoGain = CG.dayMaxAutoGain; - CG.currentAutoGain = CG.dayAutoGain; - CG.myModeMeanSetting.currentMean = CG.myModeMeanSetting.dayMean; - if (CG.isCooledCamera) - { - CG.currentEnableCooler = CG.dayEnableCooler; - CG.currentTargetTemp = CG.dayTargetTemp; - } - CG.currentTuningFile = CG.dayTuningFile; + CG.currentAutoAWB = CG.dayAutoAWB; + CG.currentWBR = CG.dayWBR; + CG.currentWBB = CG.dayWBB; + } + CG.currentDelay_ms = CG.dayDelay_ms; + CG.currentBin = CG.dayBin; + CG.currentGain = CG.dayGain; + CG.currentMaxAutoGain = CG.dayMaxAutoGain; + CG.currentAutoGain = CG.dayAutoGain; + CG.myModeMeanSetting.currentMean = CG.myModeMeanSetting.dayMean; + CG.myModeMeanSetting.currentMean_threshold = CG.myModeMeanSetting.dayMean_threshold; + if (CG.isCooledCamera) + { + CG.currentEnableCooler = CG.dayEnableCooler; + CG.currentTargetTemp = CG.dayTargetTemp; } + CG.currentTuningFile = CG.dayTuningFile; } else // NIGHT @@ -681,6 +681,7 @@ int main(int argc, char *argv[]) CG.currentMaxAutoGain = CG.nightMaxAutoGain; CG.currentAutoGain = CG.nightAutoGain; CG.myModeMeanSetting.currentMean = CG.myModeMeanSetting.nightMean; + CG.myModeMeanSetting.currentMean_threshold = CG.myModeMeanSetting.nightMean_threshold; if (CG.isCooledCamera) { CG.currentEnableCooler = CG.nightEnableCooler; @@ -694,7 +695,6 @@ int main(int argc, char *argv[]) if (CG.myModeMeanSetting.currentMean > 0.0) { CG.myModeMeanSetting.modeMean = true; - myModeMeanSetting.meanValue = CG.myModeMeanSetting.currentMean; if (! aegInit(CG, myRaspistillSetting, myModeMeanSetting)) { closeUp(EXIT_ERROR_STOP); @@ -805,7 +805,7 @@ myModeMeanSetting.modeMean = CG.myModeMeanSetting.modeMean; if (CG.lastMean == -1) { - Log(-1, "ERROR: aegCalcMean() returned mean of -1.\n"); + Log(-1, "*** %s: ERROR: aegCalcMean() returned mean of -1.\n", CG.ME); Log(2, " > Sleeping from failed exposure: %.1f seconds\n", (float)CG.currentDelay_ms / MS_IN_SEC); usleep(CG.currentDelay_ms * US_IN_MS); continue; @@ -841,7 +841,8 @@ myModeMeanSetting.modeMean = CG.myModeMeanSetting.modeMean; // Do not save this frame or sleep after it. // We just started taking images so no need to check if DAY or NIGHT changed if (remove(CG.fullFilename) != 0) - Log(0, "ERROR: Unable to remove '%s': %s\n", CG.fullFilename, strerror(errno)); + Log(0, "*** %s: ERROR: Unable to remove '%s': %s\n", + CG.ME, CG.fullFilename, strerror(errno)); continue; } else @@ -872,7 +873,7 @@ myModeMeanSetting.modeMean = CG.myModeMeanSetting.modeMean; numErrors++; if (numErrors >= maxErrors) { - Log(0, "*** ERROR: maximum number of consecutive errors of %d reached; capture program stopped.\n", maxErrors); + Log(0, "*** %s: ERROR: maximum number of consecutive errors of %d reached; capture program stopped.\n", CG.ME, maxErrors); Log(0, "Make sure cable between camera and Pi is all the way in.\n"); closeUp(EXIT_ERROR_STOP); } diff --git a/src/capture_ZWO.cpp b/src/capture_ZWO.cpp index 0a9db1656..c46525110 100644 --- a/src/capture_ZWO.cpp +++ b/src/capture_ZWO.cpp @@ -25,13 +25,7 @@ config CG; #define IS_ZWO #include "ASI_functions.cpp" -#define USE_HISTOGRAM // use the histogram code as a workaround to ZWO's bug - -#ifdef USE_HISTOGRAM -// Got these by trial and error. They are more-or-less half the max of 255, plus or minus some. -#define MINMEAN 122 -#define MAXMEAN 134 -#endif +bool useSnapshotMode = false; // XXXXXX use the ZWO snapshot exposure mode or vide mode? // Forward definitions char *getRetCode(ASI_ERROR_CODE); @@ -85,7 +79,8 @@ ASI_ERROR_CODE setControl(int camNum, ASI_CONTROL_TYPE control, long value, ASI_ ret = ASIGetControlCaps(camNum, i, &ControlCaps); if (ret != ASI_SUCCESS) { - Log(-1, "WARNING: ASIGetControlCaps() for control %d failed: %s, camNum=%d, iNumOfCtrl=%d, control=%d\n", i, getRetCode(ret), camNum, iNumOfCtrl, (int) control); + Log(-1, "*** %s: WARNING: ASIGetControlCaps() for control %d failed: %s, camNum=%d, iNumOfCtrl=%d, control=%d\n", + CG.ME, i, getRetCode(ret), camNum, iNumOfCtrl, (int) control); return(ret); } @@ -95,26 +90,31 @@ ASI_ERROR_CODE setControl(int camNum, ASI_CONTROL_TYPE control, long value, ASI_ { if (value > ControlCaps.MaxValue) { - Log(1, "WARNING: Value of %ld greater than max value allowed (%ld) for control '%s' (#%d).\n", value, ControlCaps.MaxValue, ControlCaps.Name, ControlCaps.ControlType); + Log(1, "*** %s: WARNING: Value of %ld greater than max value allowed (%ld) for control '%s' (#%d).\n", + CG.ME, value, ControlCaps.MaxValue, ControlCaps.Name, ControlCaps.ControlType); value = ControlCaps.MaxValue; } else if (value < ControlCaps.MinValue) { - Log(1, "WARNING: Value of %ld less than min value allowed (%ld) for control '%s' (#%d).\n", value, ControlCaps.MinValue, ControlCaps.Name, ControlCaps.ControlType); + Log(1, "*** %s: WARNING: Value of %ld less than min value allowed (%ld) for control '%s' (#%d).\n", + CG.ME, value, ControlCaps.MinValue, ControlCaps.Name, ControlCaps.ControlType); value = ControlCaps.MinValue; } if (makeAuto == ASI_TRUE && ControlCaps.IsAutoSupported == ASI_FALSE) { - Log(1, "WARNING: control '%s' (#%d) doesn't support auto mode.\n", ControlCaps.Name, ControlCaps.ControlType); + Log(1, "*** %s: WARNING: control '%s' (#%d) doesn't support auto mode.\n", + CG.ME, ControlCaps.Name, ControlCaps.ControlType); makeAuto = ASI_FALSE; } ret = ASISetControlValue(camNum, control, value, makeAuto); if (ret != ASI_SUCCESS) { - Log(-1, "WARNING: ASISetControlCaps() for control %d, value=%ld failed: %s\n", control, value, getRetCode(ret)); + Log(-1, "*** %s: WARNING: ASISetControlCaps() for control %d, value=%ld failed: %s\n", + CG.ME, control, value, getRetCode(ret)); return(ret); } } else { - Log(0, "ERROR: ControlCap: '%s' (#%d) not writable; not setting to %ld.\n", ControlCaps.Name, ControlCaps.ControlType, value); + Log(0, "*** %s: ERROR: ControlCap: '%s' (#%d) not writable; not setting to %ld.\n", + CG.ME, ControlCaps.Name, ControlCaps.ControlType, value); ret = ASI_ERROR_INVALID_MODE; // this seemed like the closest error } return ret; @@ -171,7 +171,7 @@ void *SaveImgThd(void *para) if (pRgb.data) { char cmd[1100+strlen(CG.allskyHome)]; - Log(1, " > Saving %s image '%s'\n", CG.takeDarkFrames ? "dark" : dayOrNight.c_str(), CG.finalFileName); + Log(4, " > Saving %s image '%s'\n", CG.takeDarkFrames ? "dark" : dayOrNight.c_str(), CG.finalFileName); snprintf(cmd, sizeof(cmd), "%s/scripts/saveImage.sh %s '%s'", CG.allskyHome, dayOrNight.c_str(), CG.fullFilename); add_variables_to_command(CG, cmd, exposureStartDateTime); strcat(cmd, " &"); @@ -183,14 +183,14 @@ void *SaveImgThd(void *para) } catch (const cv::Exception& ex) { - Log(0, "*** ERROR: Exception saving image: %s\n", ex.what()); + Log(0, "*** %s: ERROR: Exception saving image: %s\n", CG.ME, ex.what()); } et = std::chrono::high_resolution_clock::now(); if (result) system(cmd); else - Log(0, "*** ERROR: Unable to save image '%s'.\n", CG.fullFilename); + Log(0, "*** %s: ERROR: Unable to save image '%s'.\n", CG.ME, CG.fullFilename); } else { // This can happen if the program is closed before the first picture. @@ -212,7 +212,7 @@ void *SaveImgThd(void *para) x = " > *****\n"; // indicate when it takes a REALLY long time to save else x = ""; - Log(3, "%s > Image took %'.1f ms to save (average %'.1f ms).\n%s", x, diff_ms, totalTime_ms / totalSaves, x); + Log(4, "%s > Image took %'.1f ms to save (average %'.1f ms).\n%s", x, diff_ms, totalTime_ms / totalSaves, x); } pthread_mutex_unlock(&mtxSaveImg); @@ -221,14 +221,7 @@ void *SaveImgThd(void *para) return (void *)0; } -long roundTo(long n, int roundTo) -{ - long a = (n / roundTo) * roundTo; // Smaller multiple - long b = a + roundTo; // Larger multiple - return (n - a > b - n)? b : a; // Return of closest of two -} -#ifdef USE_HISTOGRAM // As of July 2021, ZWO's SDK (version 1.9) has a bug where autoexposure daylight shots' // exposures jump all over the place. One is way too dark and the next way too light, etc. // As a workaround, our histogram code replaces ZWO's code auto-exposure mechanism. @@ -238,26 +231,35 @@ long roundTo(long n, int roundTo) // eg. box size 0x0, box size WxW, box crosses image edge, ... basically // anything that would read/write out-of-bounds -int computeHistogram(unsigned char *imageBuffer, config cg, int *histogram) +int computeHistogram(unsigned char *imageBuffer, config cg, bool useHistogramBox) { - int h, i; unsigned char *buf = imageBuffer; + int histogram[256]; // Clear the histogram array. - for (h = 0; h < 256; h++) { - histogram[h] = 0; + for (int i = 0; i < 256; i++) { + histogram[i] = 0; } // Different image types have a different number of bytes per pixel. cg.width *= currentBpp; - int roiX1 = (cg.width * cg.HB.histogramBoxPercentFromLeft) - (cg.HB.currentHistogramBoxSizeX * currentBpp / 2); - int roiX2 = roiX1 + (currentBpp * cg.HB.currentHistogramBoxSizeX); - int roiY1 = (cg.height * cg.HB.histogramBoxPercentFromTop) - (cg.HB.currentHistogramBoxSizeY / 2); - int roiY2 = roiY1 + cg.HB.currentHistogramBoxSizeY; - - // Start off and end on a logical pixel boundries. - roiX1 = (roiX1 / currentBpp) * currentBpp; - roiX2 = (roiX2 / currentBpp) * currentBpp; + int roiX1, roiX2, roiY1, roiY2; + if (useHistogramBox) + { + roiX1 = (cg.width * cg.HB.histogramBoxPercentFromLeft) - (cg.HB.currentHistogramBoxSizeX * currentBpp / 2); + roiX2 = roiX1 + (currentBpp * cg.HB.currentHistogramBoxSizeX); + roiY1 = (cg.height * cg.HB.histogramBoxPercentFromTop) - (cg.HB.currentHistogramBoxSizeY / 2); + roiY2 = roiY1 + cg.HB.currentHistogramBoxSizeY; + + // Start off and end on a logical pixel boundries. + roiX1 = (roiX1 / currentBpp) * currentBpp; + roiX2 = (roiX2 / currentBpp) * currentBpp; + } else { + roiX1 = 0; + roiX2 = cg.width; + roiY1 = 0; + roiY2 = cg.height; + } // For RGB24, data for each pixel is stored in 3 consecutive bytes: blue, green, red. // For all image types, each row in the image contains one row of pixels. @@ -268,7 +270,7 @@ int computeHistogram(unsigned char *imageBuffer, config cg, int *histogram) case IMG_Y8: for (int y = roiY1; y < roiY2; y++) { for (int x = roiX1; x < roiX2; x+=currentBpp) { - i = (cg.width * y) + x; + int i = (cg.width * y) + x; int total = 0; for (int z = 0; z < currentBpp; z++) { @@ -283,7 +285,7 @@ int computeHistogram(unsigned char *imageBuffer, config cg, int *histogram) case IMG_RAW16: for (int y = roiY1; y < roiY2; y++) { for (int x = roiX1; x < roiX2; x+=currentBpp) { - i = (cg.width * y) + x; + int i = (cg.width * y) + x; int pixelValue; // This assumes the image data is laid out in big endian format. // We are going to grab the most significant byte @@ -302,9 +304,9 @@ int computeHistogram(unsigned char *imageBuffer, config cg, int *histogram) // Now calculate the mean. int meanBin = 0; int a = 0, b = 0; - for (int h = 0; h < 256; h++) { - a += (h+1) * histogram[h]; - b += histogram[h]; + for (int i = 0; i < 256; i++) { + a += (i+1) * histogram[i]; + b += histogram[i]; } if (b == 0) @@ -316,26 +318,30 @@ int computeHistogram(unsigned char *imageBuffer, config cg, int *histogram) meanBin = a/b - 1; return meanBin; } -#endif // This is based on code from PHD2. -// Camera has 2 internal frame buffers we need to clear. +// Camera has internal frame buffers we need to clear. // The camera and/or driver will buffer frames and return the oldest one which // could be very old. Read out all the buffered frames so the frame we get is current. -void flushBufferedImages(int cameraId, void *buf, size_t size) +void flushBufferedImages(config *cg, void *buf, size_t size) { enum { NUM_IMAGE_BUFFERS = 2 }; + ASI_ERROR_CODE status; + + setControl(cg->cameraNumber, ASI_EXPOSURE, cg->cameraMinExposure_us, ASI_FALSE); - int numCleared; - for (numCleared = 0; numCleared < NUM_IMAGE_BUFFERS; numCleared++) + for (int i = 0; i < NUM_IMAGE_BUFFERS; i++) { - ASI_ERROR_CODE status = ASIGetVideoData(cameraId, (unsigned char *) buf, size, 1); -/*xxxxx - if (status != ASI_SUCCESS) - break; // no more buffered frames -*/ - if (status != ASI_ERROR_TIMEOUT) // Most are ASI_ERROR_TIMEOUT, so don't show them + status = ASIGetVideoData(cg->cameraNumber, (unsigned char *) buf, size, 1); + if (status == ASI_SUCCESS) + { Log(3, " > [Cleared buffer frame]: %s\n", getRetCode(status)); + } + else if (status != ASI_ERROR_TIMEOUT) + { + Log(0, "*** %s: ERROR: flushBufferedImages() got %s\n", cg->ME, getRetCode(status)); + } + // TODO: in theory if status == ASI_ERROR_TIMEOUT we could stop. } } @@ -349,13 +355,14 @@ ASI_BOOL bAuto = ASI_FALSE; ASI_BOOL wasAutoExposure = ASI_FALSE; long bufferSize = NOT_SET; -ASI_ERROR_CODE takeOneExposure(config *cg, unsigned char *imageBuffer, int *histogram) +ASI_ERROR_CODE takeOneExposure(config *cg, unsigned char *imageBuffer) { if (imageBuffer == NULL) { return (ASI_ERROR_CODE) -1; } - ASI_ERROR_CODE status; + ASI_ERROR_CODE status, ret; + // ZWO recommends timeout = (exposure*2) + 500 ms // After some discussion, we're doing +5000ms to account for delays induced by // USB contention, such as that caused by heavy USB disk IO @@ -363,15 +370,21 @@ ASI_ERROR_CODE takeOneExposure(config *cg, unsigned char *imageBuffer, int *hist // This debug message isn't typcally needed since we already displayed a message about // starting a new exposure, and below we display the result when the exposure is done. - Log(4, " > %s to %s\n", - wasAutoExposure == ASI_TRUE ? "Camera set auto-exposure" : "Exposure set", + Log(3, " > %s to %s\n", + cg->HB.useHistogram ? "Histogram set exposure" : + (wasAutoExposure == ASI_TRUE ? "Camera set auto-exposure" : "Manual exposure set"), length_in_units(cg->currentExposure_us, true)); - setControl(cg->cameraNumber, ASI_EXPOSURE, cg->currentExposure_us, cg->currentAutoExposure ? ASI_TRUE :ASI_FALSE); + if (! useSnapshotMode) + flushBufferedImages(cg, imageBuffer, bufferSize); + + // Sanity check. + if (cg->HB.useHistogram && cg->currentAutoExposure == ASI_TRUE) + Log(0, "*** %s: ERROR: HB.useHistogram AND currentAutoExposure are both set\n", cg->ME); - flushBufferedImages(cg->cameraNumber, imageBuffer, bufferSize); + setControl(cg->cameraNumber, ASI_EXPOSURE, cg->currentExposure_us, cg->currentAutoExposure ? ASI_TRUE : ASI_FALSE); - if (cg->videoOffBetweenImages) + if (! useSnapshotMode && cg->videoOffBetweenImages) { status = ASIStartVideoCapture(cg->cameraNumber); } else { @@ -382,14 +395,25 @@ ASI_ERROR_CODE takeOneExposure(config *cg, unsigned char *imageBuffer, int *hist // Make sure the actual time to take the picture is "close" to the requested time. auto tStart = std::chrono::high_resolution_clock::now(); - status = ASIGetVideoData(cg->cameraNumber, imageBuffer, bufferSize, timeout); - if (cg->videoOffBetweenImages) - (void) ASIStopVideoCapture(cg->cameraNumber); + if (useSnapshotMode) + { +// xxxxxxxxxxxxxxxxxx start exposure, sleep for 95% of cg->currentExposure_us, then check every 5 us. + } else { + status = ASIGetVideoData(cg->cameraNumber, imageBuffer, bufferSize, timeout); + if (cg->videoOffBetweenImages) + { + ret = ASIStopVideoCapture(cg->cameraNumber); + if (ret != ASI_SUCCESS) + { + Log(1, " > %s: WARNING: ASIStopVideoCapture() failed: %s\n", cg->ME, getRetCode(ret)); + } + } + } if (status != ASI_SUCCESS) { int exitCode; - Log(0, " > ERROR: Failed getting image: %s\n", getRetCode(status)); + Log(0, " > %s: ERROR: Failed getting image: %s\n", cg->ME, getRetCode(status)); // Check if we reached the maximum number of consective errors if (! checkMaxErrors(&exitCode, maxErrors)) @@ -419,77 +443,109 @@ ASI_ERROR_CODE takeOneExposure(config *cg, unsigned char *imageBuffer, int *hist // so only check for long ones. // Testing shows there's about this much us overhead, // so subtract it to get our best estimate of the "actual" time. - const int OVERHEAD = 340000; + const int OVERHEAD_us = (int) (0.34 * US_IN_SEC); // Don't subtract if it would have made timeToTakeImage_us negative. - if (timeToTakeImage_us > OVERHEAD) - diff_us -= OVERHEAD; + if (timeToTakeImage_us > OVERHEAD_us) + diff_us -= OVERHEAD_us; threshold_us = cg->currentExposure_us * 0.5; // 50% seems like a good number if (abs(diff_us) > threshold_us) tooShort = true; } + if (tooShort) { - Log(1, "*** WARNING: Time to take exposure (%s) ", + Log(1, " *** WARNING: Time to take exposure (%s) ", length_in_units(timeToTakeImage_us, true)); - Log(1, "differs from requested exposure time (%s) ", - length_in_units(cg->currentExposure_us, true)); - Log(1, "by %s, ", length_in_units(diff_us, true)); - Log(1, "threshold=%'ld\n", length_in_units(threshold_us, true)); + Log(1, "differs from requested exposure time (%s) by %s, threshold=%s\n", + length_in_units(cg->currentExposure_us, true), + length_in_units(diff_us, true), + length_in_units(threshold_us, true)); } else { - Log(4, " > timeToTakeImage_us=%'ld us, diff_us=%'ld, threshold_us=%'ld\n", timeToTakeImage_us, diff_us, threshold_us); +// XXXXXXXXXXXXX set to 4 after testing + Log(3, " > Time to take exposure=%'ld us, diff_us=%'ld", timeToTakeImage_us, diff_us); + if (threshold_us > 0) + Log(3, ", threshold_us=%'ld", threshold_us); + Log(3, "\n"); } numErrors = 0; long l; - ASIGetControlValue(cg->cameraNumber, ASI_GAIN, &l, &bAuto); + ret = ASIGetControlValue(cg->cameraNumber, ASI_GAIN, &l, &bAuto); + if (ret != ASI_SUCCESS) + { + Log(1, " > %s: WARNING: ASIGetControlValue(ASI_GAIN) failed: %s\n", cg->ME, getRetCode(ret)); + } cg->lastGain = (double) l; - debug_text[0] = '\0'; + char tempBuf[500]; + tempBuf[0] = '\0'; + char *tb = tempBuf; -#ifdef USE_HISTOGRAM - if (histogram != NULL) - { - cg->lastMean = (double)computeHistogram(imageBuffer, *cg, histogram); + cg->lastMean = (double)computeHistogram(imageBuffer, *cg, true); - sprintf(debug_text, " @ mean %d", (int) cg->lastMean); - if (cg->currentAutoGain && ! cg->takeDarkFrames) - { - char *p = debug_text + strlen(debug_text); - sprintf(p, ", auto gain %ld", (long) cg->lastGain); - } - } -#endif +// xxxxxx for testing. Get the mean of the whole image so we can compare to what removeBadImages.sh calculates. +// If it's the same, then the algorithms are the same and removeBadImages.sh can use MEAN. +cg->lastMeanFull = (double)computeHistogram(imageBuffer, *cg, false); + + sprintf(tb, " @ mean %d, %sgain %ld, fullMean %d", + (int) cg->lastMean, cg->currentAutoGain ? "(auto) " : "", + (long) cg->lastGain, (int) cg->lastMeanFull); cg->lastExposure_us = cg->currentExposure_us; + // Per ZWO, when in manual-exposure mode, the returned exposure length should always // be equal to the requested length; in fact, "there's no need to call ASIGetControlValue()". // When in auto-exposure mode, the returned exposure length is what the driver thinks the // next exposure should be, and will eventually converge on the correct exposure. - ASIGetControlValue(cg->cameraNumber, ASI_EXPOSURE, &suggestedNextExposure_us, &wasAutoExposure); - Log(2, " > Got image%s.", debug_text); - if (cg->currentAutoExposure) Log(3, " Suggested next exposure: %s", length_in_units(suggestedNextExposure_us, true)); + ret = ASIGetControlValue(cg->cameraNumber, ASI_EXPOSURE, &suggestedNextExposure_us, &wasAutoExposure); + if (ret != ASI_SUCCESS) + { + Log(1, " > WARNING: ASIGetControlValue(ASI_EXPOSURE) failed: %s\n", cg->ME, getRetCode(ret)); + } + Log(2, " > GOT IMAGE%s.", tb); + Log(3, cg->HB.useHistogram ? " Ignoring suggested next exposure of %s." : " Suggested next exposure: %s.", + length_in_units(suggestedNextExposure_us, true)); Log(2, "\n"); long temp; - ASIGetControlValue(cg->cameraNumber, ASI_TEMPERATURE, &temp, &bAuto); + ret = ASIGetControlValue(cg->cameraNumber, ASI_TEMPERATURE, &temp, &bAuto); + if (ret != ASI_SUCCESS) + { + Log(1, " > %s: WARNING: ASIGetControlValue(ASI_TEMPERATURE) failed: %s\n", cg->ME, getRetCode(ret)); + } cg->lastSensorTemp = (long) ((double)temp / cg->divideTemperatureBy); if (cg->isColorCamera) { - ASIGetControlValue(cg->cameraNumber, ASI_WB_R, &l, &bAuto); + ret = ASIGetControlValue(cg->cameraNumber, ASI_WB_R, &l, &bAuto); + if (ret != ASI_SUCCESS) + { + Log(1, " > %s: WARNING: ASIGetControlValue(ASI_WB_R) failed: %s\n", cg->ME, getRetCode(ret)); + } cg->lastWBR = (double) l; - ASIGetControlValue(cg->cameraNumber, ASI_WB_B, &l, &bAuto); + + ret = ASIGetControlValue(cg->cameraNumber, ASI_WB_B, &l, &bAuto); + if (ret != ASI_SUCCESS) + { + Log(1, " > %s: WARNING: ASIGetControlValue(ASI_WB_B) failed: %s\n", cg->ME, getRetCode(ret)); + } cg->lastWBB = (double) l; } if (cg->asiAutoBandwidth) - ASIGetControlValue(cg->cameraNumber, ASI_BANDWIDTHOVERLOAD, &cg->lastAsiBandwidth, &wasAutoExposure); + { + ret = ASIGetControlValue(cg->cameraNumber, ASI_BANDWIDTHOVERLOAD, &cg->lastAsiBandwidth, &wasAutoExposure); + if (ret != ASI_SUCCESS) + { + Log(1, " > %s: WARNING: ASIGetControlValue(ASI_BANDWIDTHOVERLOAD) failed: %s\n", cg->ME, getRetCode(ret)); + } + } } } else { - Log(0, " > ERROR: Not fetching exposure data because status is %s\n", getRetCode(status)); + Log(0, " > %s: ERROR: Not fetching exposure data because status is %s\n", cg->ME, getRetCode(status)); } return status; @@ -573,7 +629,7 @@ int determineGainChange(config cg) if (numGainChanges > gainTransitionImages || totalAdjustGain == 0) { // no more changes needed in this transition - Log(4, " xxxx No more gain changes needed.\n"); + Log(4, " No more gain changes needed.\n"); currentAdjustGain = false; return(0); } @@ -621,7 +677,7 @@ bool checkMaxErrors(int *e, int maxErrors) if (numErrors >= maxErrors) { *e = EXIT_RESET_USB; // exit code. Need to reset USB bus - Log(0, "*** ERROR: Maximum number of consecutive errors of %d reached; capture program exited.\n", maxErrors); + Log(0, "*** %s: ERROR: Maximum number of consecutive errors of %d reached; capture program exited.\n", CG.ME, maxErrors); return(false); // gets us out of inner and outer loop } return(true); @@ -637,7 +693,7 @@ int main(int argc, char *argv[]) static char *a = getenv("ALLSKY_HOME"); // This must come before anything else if (a == NULL) { - Log(0, "%s: ERROR: ALLSKY_HOME not set!\n", CG.ME); + Log(0, "*** %s: ERROR: ALLSKY_HOME not set!\n", CG.ME); exit(EXIT_ERROR_STOP); } else @@ -662,7 +718,6 @@ int main(int argc, char *argv[]) // In theory, almost every setting could have both day and night versions (e.g., width & height), // but the chances of someone wanting different versions. -#ifdef USE_HISTOGRAM int maxHistogramAttempts = 15; // max number of times we'll try for a better histogram mean // If we just transitioned from night to day, it's possible currentExposure_us will @@ -676,61 +731,61 @@ int main(int argc, char *argv[]) // Note that it's likely getting lighter outside with every exposure // so the mean will eventually get into the valid range. const int percentChange = 10.0; // percent of ORIGINAL difference -#endif - //------------------------------------------------------------------------------------------------------- - //------------------------------------------------------------------------------------------------------- - setlinebuf(stdout); // Line buffer output so entries appear in the log immediately. + //--------------------------------------------------------------------------------------------- + //--------------------------------------------------------------------------------------------- + setlinebuf(stdout); // Line buffer output so entries appear in the log immediately. CG.ct = ctZWO; - if (! getCommandLineArguments(&CG, argc, argv)) - { - // getCommandLineArguents outputs an error message. - exit(EXIT_ERROR_STOP); - } - if (! CG.saveCC && ! CG.help) - { - displayHeader(CG); - } + processConnectedCameras(); // exits on error. Sets CG.cameraNumber. - doLocale(&CG); - - if (CG.help) - { - displayHelp(CG); - closeUp(EXIT_OK); - } - - processConnectedCameras(); // exits on error - - ASI_CAMERA_INFO ASICameraInfo; asiRetCode = ASIOpenCamera(CG.cameraNumber); if (asiRetCode != ASI_SUCCESS) { - Log(0, "*** ERROR opening camera, check that you have root permissions! (%s)\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: opening camera, check that you have root permissions! (%s)\n", + CG.ME, getRetCode(asiRetCode)); closeUp(EXIT_NO_CAMERA); } + ASI_CAMERA_INFO ASICameraInfo; asiRetCode = ASIGetCameraProperty(&ASICameraInfo, CG.cameraNumber); if (asiRetCode != ASI_SUCCESS) { - Log(0, "ERROR: ASIGetCamerProperty() returned: %s\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: ASIGetCamerProperty() returned: %s\n", CG.ME, getRetCode(asiRetCode)); exit(EXIT_ERROR_STOP); } asiRetCode = ASIGetNumOfControls(CG.cameraNumber, &iNumOfCtrl); if (asiRetCode != ASI_SUCCESS) { - Log(0, "ERROR: ASIGetNumOfControls() returned: %s\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: ASIGetNumOfControls() returned: %s\n", CG.ME, getRetCode(asiRetCode)); exit(EXIT_ERROR_STOP); } CG.ASIversion = ASIGetSDKVersion(); - // Set defaults that depend on the camera type. if (! setDefaults(&CG, ASICameraInfo)) closeUp(EXIT_ERROR_STOP); + if (! getCommandLineArguments(&CG, argc, argv)) + { + // getCommandLineArguents outputs an error message. + exit(EXIT_ERROR_STOP); + } + + if (! CG.saveCC && ! CG.help) + { + displayHeader(CG); + } + + doLocale(&CG); + + if (CG.help) + { + displayHelp(CG); + closeUp(EXIT_OK); + } + // Do argument error checking if we're not going to exit soon. if (! CG.saveCC && ! validateSettings(&CG, ASICameraInfo)) closeUp(EXIT_ERROR_STOP); @@ -776,7 +831,6 @@ int main(int argc, char *argv[]) // checkExposureValues() must come after outputCameraInfo(). (void) checkExposureValues(&CG); -#ifdef USE_HISTOGRAM // The histogram box needs to fit on the image. // If we're binning we'll decrease the size of the box accordingly. bool ok = true; @@ -784,19 +838,19 @@ int main(int argc, char *argv[]) { if (sscanf(CG.HB.sArgs, "%d %d %f %f", &CG.HB.histogramBoxSizeX, &CG.HB.histogramBoxSizeY, &CG.HB.histogramBoxPercentFromLeft, &CG.HB.histogramBoxPercentFromTop) != 4) { - Log(0, "%s*** ERROR: Not enough histogram box parameters should be 4: '%s'%s\n", c(KRED), CG.HB.sArgs, c(KNRM)); + Log(0, "*** %s: ERROR: Not enough histogram box parameters should be 4: '%s'\n", CG.ME, CG.HB.sArgs); ok = false; } else { if (CG.HB.histogramBoxSizeX < 1 || CG.HB.histogramBoxSizeY < 1) { - Log(0, "%s*** ERROR: Histogram box size must be > 0; you entered X=%d, Y=%d%s\n", - c(KRED), CG.HB.histogramBoxSizeX, CG.HB.histogramBoxSizeY, c(KNRM)); + Log(0, "*** %s: ERROR: Histogram box size must be > 0; you entered X=%d, Y=%d\n", + CG.ME, CG.HB.histogramBoxSizeX, CG.HB.histogramBoxSizeY); ok = false; } if (CG.HB.histogramBoxPercentFromLeft < 0.0 || CG.HB.histogramBoxPercentFromTop < 0.0) { - Log(0, "%s*** ERROR: Histogram box percents must be > 0; you entered X=%.0f%%, Y=%.0f%%%s\n", - c(KRED), (CG.HB.histogramBoxPercentFromLeft*100.0), (CG.HB.histogramBoxPercentFromTop*100.0), c(KNRM)); + Log(0, "*** %s: ERROR: Histogram box percents must be > 0; you entered X=%.0f%%, Y=%.0f%%\n", + CG.ME, (CG.HB.histogramBoxPercentFromLeft*100.0), (CG.HB.histogramBoxPercentFromTop*100.0)); ok = false; } else @@ -816,13 +870,14 @@ int main(int argc, char *argv[]) if (CG.HB.leftOfBox < 0 || CG.HB.rightOfBox >= CG.width || CG.HB.topOfBox < 0 || CG.HB.bottomOfBox >= CG.height) { - Log(0, "%s*** ERROR: Histogram box location must fit on image; upper left of box is %dx%d, lower right %dx%d%s\n", c(KRED), CG.HB.leftOfBox, CG.HB.topOfBox, CG.HB.rightOfBox, CG.HB.bottomOfBox, c(KNRM)); + Log(0, "*** %s: ERROR: Histogram box location must fit on image; upper left of box is %dx%d, lower right %dx%d%s\n", + CG.ME, CG.HB.leftOfBox, CG.HB.topOfBox, CG.HB.rightOfBox, CG.HB.bottomOfBox); ok = false; } // else everything is hunky dory } } } else { - Log(0, "%s*** ERROR: No values specified for histogram box%s\n", c(KRED), c(KNRM)); + Log(0, "*** %s: ERROR: No values specified for histogram box.\n", CG.ME); ok = false; } @@ -830,12 +885,11 @@ int main(int argc, char *argv[]) { closeUp(EXIT_ERROR_STOP); // force the user to fix it } -#endif asiRetCode = ASIInitCamera(CG.cameraNumber); if (asiRetCode != ASI_SUCCESS) { - Log(0, "*** ERROR: Unable to initialise camera: %s\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: Unable to initialise camera: %s\n", CG.ME, getRetCode(asiRetCode)); closeUp(EXIT_ERROR_STOP); // Can't do anything so might as well exit. } @@ -888,7 +942,7 @@ int main(int argc, char *argv[]) } else { - Log(0, "*** ERROR: Unknown Image Type: %d\n", CG.imageType); + Log(0, "*** %s: ERROR: Unknown Image Type: %d\n", CG.ME, CG.imageType); closeUp(EXIT_ERROR_STOP); } @@ -950,7 +1004,7 @@ int main(int argc, char *argv[]) asiRetCode = ASIStartVideoCapture(CG.cameraNumber); if (asiRetCode != ASI_SUCCESS) { - Log(0, "*** ERROR: Unable to start video capture: %s\n", getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: Unable to start video capture: %s\n", CG.ME, getRetCode(asiRetCode)); closeUp(EXIT_ERROR_STOP); } } @@ -973,7 +1027,10 @@ int main(int argc, char *argv[]) CG.nightAutoExposure = false; CG.currentAutoGain = false; CG.currentGain = CG.nightGain; - CG.currentMaxAutoGain = CG.nightMaxAutoGain; // not needed since we're not using auto gain, but set to be consistent + + // not needed since we're not using auto gain, but set to be consistent + CG.currentMaxAutoGain = CG.nightMaxAutoGain; + gainChange = 0; CG.currentDelay_ms = CG.nightDelay_ms; CG.currentMaxAutoExposure_us = CG.currentExposure_us = CG.nightMaxAutoExposure_us; @@ -991,6 +1048,7 @@ int main(int argc, char *argv[]) CG.currentTargetTemp = CG.nightTargetTemp; } CG.myModeMeanSetting.currentMean = NOT_SET; + CG.myModeMeanSetting.currentMean_threshold = NOT_SET; CG.myModeMeanSetting.modeMean = false; CG.HB.useHistogram = false; @@ -1021,92 +1079,85 @@ int main(int argc, char *argv[]) continue; } + Log(1, "==========\n=== Starting daytime capture ===\n==========\n"); + + // We only skip initial frames if we are starting in daytime and using auto-exposure. + if (numExposures == 0 && CG.dayAutoExposure) + CG.currentSkipFrames = CG.daySkipFrames; + + // If we went from Night to Day, then currentExposure_us will be the last night + // exposure so leave it if we're using auto-exposure so there's a seamless change from + // Night to Day, i.e., if the exposure was fine a minute ago it will likely be fine now. + // On the other hand, if this program just started or we're using manual exposures, + // use what the user specified. + if (numExposures == 0 || ! CG.dayAutoExposure) + { + CG.currentExposure_us = CG.dayExposure_us; + } else { - Log(1, "==========\n=== Starting daytime capture ===\n==========\n"); - - // We only skip initial frames if we are starting in daytime and using auto-exposure. - if (numExposures == 0 && CG.dayAutoExposure) - CG.currentSkipFrames = CG.daySkipFrames; - - // If we went from Night to Day, then currentExposure_us will be the last night - // exposure so leave it if we're using auto-exposure so there's a seamless change from - // Night to Day, i.e., if the exposure was fine a minute ago it will likely be fine now. - // On the other hand, if this program just started or we're using manual exposures, - // use what the user specified. - if (numExposures == 0 || ! CG.dayAutoExposure) - { - CG.currentExposure_us = CG.dayExposure_us; - } - else - { - // If gain changes, we have to change the exposure time to get an equally - // exposed image. - // ZWO gain has unit 0.1dB, so we have to convert the gain values to a factor first - // newExp = (oldExp * oldGain) / newGain - // e.g. 20s = (10s * 2.0) / (1.0) - - // current values here are last night's values - double oldGain = pow(10, CG.currentGain / 10.0 / 20.0); - double newGain = pow(10, CG.dayGain / 10.0 / 20.0); - Log(4, "Using the last night exposure (%s),", length_in_units(CG.currentExposure_us, true)); - CG.currentExposure_us = (CG.currentExposure_us * oldGain) / newGain; - Log(4," old (%'2f) and new (%'2f) Gain to calculate new exposure of %s\n", oldGain, newGain, length_in_units(CG.currentExposure_us, true)); - } + // If gain changes, we have to change the exposure time to get an equally + // exposed image. + // ZWO gain has unit 0.1dB, so we have to convert the gain values to a factor first + // newExp = (oldExp * oldGain) / newGain + // e.g. 20s = (10s * 2.0) / (1.0) + + // current values here are last night's values + double oldGain = pow(10, CG.currentGain / 10.0 / 20.0); + double newGain = pow(10, CG.dayGain / 10.0 / 20.0); + Log(3, "Using the last night exposure (%s),", length_in_units(CG.currentExposure_us, true)); + CG.currentExposure_us = (CG.currentExposure_us * oldGain) / newGain; + Log(3," old (%'2f) and new (%'2f) Gain to calculate new exposure of %s\n", + oldGain, newGain, length_in_units(CG.currentExposure_us, true)); + } - CG.currentMaxAutoExposure_us = CG.dayMaxAutoExposure_us; - Log(4, "currentMaxAutoExposure_us set to daytime value of %s.\n", length_in_units(CG.currentMaxAutoExposure_us, true)); - if (CG.currentExposure_us > CG.currentMaxAutoExposure_us) { - Log(3, "Decreasing currentExposure_us from %s", length_in_units(CG.currentExposure_us, true)); - Log(3, "to %s\n", length_in_units(CG.currentMaxAutoExposure_us, true)); - CG.currentExposure_us = CG.currentMaxAutoExposure_us; - } -#ifdef USE_HISTOGRAM - // Don't use camera auto-exposure since we mimic it ourselves. - if (CG.dayAutoExposure) - { - CG.HB.useHistogram = true; - Log(4, "Turning off ZWO auto-exposure to use Allsky auto-exposure.\n"); - } - else - { - CG.HB.useHistogram = false; - } - // With the histogram method we NEVER use ZWO auto exposure - either the user said - // not to, or we turn it off ourselves. - CG.currentAutoExposure = false; -#else - CG.currentAutoExposure = CG.dayAutoExposure; - CG.HB.useHistogram = false; -#endif - CG.currentBrightness = CG.dayBrightness; - if (CG.isColorCamera) - { - CG.currentAutoAWB = CG.dayAutoAWB; - CG.currentWBR = CG.dayWBR; - CG.currentWBB = CG.dayWBB; - } - CG.currentDelay_ms = CG.dayDelay_ms; - CG.currentBin = CG.dayBin; - CG.currentGain = CG.dayGain; // must come before determineGainChange() below - CG.currentMaxAutoGain = CG.dayMaxAutoGain; - if (currentAdjustGain) - { - // we did some nightime images so adjust gain - numGainChanges = 0; - gainChange = determineGainChange(CG); - } - else - { - gainChange = 0; - } - CG.currentAutoGain = CG.dayAutoGain; - CG.myModeMeanSetting.currentMean = CG.myModeMeanSetting.dayMean; - if (CG.isCooledCamera) - { - CG.currentEnableCooler = CG.dayEnableCooler; - CG.currentTargetTemp = CG.dayTargetTemp; - } + CG.currentMaxAutoExposure_us = CG.dayMaxAutoExposure_us; + Log(3, "currentMaxAutoExposure_us set to daytime value of %s.\n", + length_in_units(CG.currentMaxAutoExposure_us, true)); + if (CG.currentExposure_us > CG.currentMaxAutoExposure_us) { + Log(3, "Decreasing currentExposure_us from %s to %s\n", + length_in_units(CG.currentExposure_us, true), + length_in_units(CG.currentMaxAutoExposure_us, true)); + CG.currentExposure_us = CG.currentMaxAutoExposure_us; + } + // Don't use camera auto-exposure since we mimic it ourselves. + CG.HB.useHistogram = CG.dayAutoExposure; + if (CG.HB.useHistogram) + { + // Only need to display this once, not every night-to-day transition... + Log(4, "Turning off daytime ZWO auto-exposure to use Allsky auto-exposure.\n"); + } + // With the histogram method we NEVER use ZWO auto exposure - either the user said + // not to, or we turn it off ourselves. + CG.currentAutoExposure = false; + CG.currentBrightness = CG.dayBrightness; + if (CG.isColorCamera) + { + CG.currentAutoAWB = CG.dayAutoAWB; + CG.currentWBR = CG.dayWBR; + CG.currentWBB = CG.dayWBB; + } + CG.currentDelay_ms = CG.dayDelay_ms; + CG.currentBin = CG.dayBin; + CG.currentGain = CG.dayGain; // must come before determineGainChange() below + CG.currentMaxAutoGain = CG.dayMaxAutoGain; + if (currentAdjustGain) + { + // we did some nightime images so adjust gain + numGainChanges = 0; + gainChange = determineGainChange(CG); + } + else + { + gainChange = 0; + } + CG.currentAutoGain = CG.dayAutoGain; + CG.myModeMeanSetting.currentMean = CG.myModeMeanSetting.dayMean; + CG.myModeMeanSetting.currentMean_threshold = CG.myModeMeanSetting.dayMean_threshold; + if (CG.isCooledCamera) + { + CG.currentEnableCooler = CG.dayEnableCooler; + CG.currentTargetTemp = CG.dayTargetTemp; } } @@ -1133,7 +1184,18 @@ int main(int argc, char *argv[]) CG.currentExposure_us = CG.nightExposure_us; } +if (CG.HB.useExperimentalExposure) { + // Don't use camera auto-exposure since we mimic it ourselves. + CG.HB.useHistogram = CG.nightAutoExposure; + if (CG.HB.useHistogram) + { + Log(4, "Turning off nighttime ZWO auto-exposure to use Allsky auto-exposure.\n"); + } + CG.currentAutoExposure = false; +} else { CG.currentAutoExposure = CG.nightAutoExposure; + CG.HB.useHistogram = false; // only used during day +} CG.currentBrightness = CG.nightBrightness; if (CG.isColorCamera) { @@ -1158,21 +1220,28 @@ int main(int argc, char *argv[]) } CG.currentAutoGain = CG.nightAutoGain; CG.myModeMeanSetting.currentMean = CG.myModeMeanSetting.nightMean; + CG.myModeMeanSetting.currentMean_threshold = CG.myModeMeanSetting.nightMean_threshold; if (CG.isCooledCamera) { CG.currentEnableCooler = CG.nightEnableCooler; CG.currentTargetTemp = CG.nightTargetTemp; } - CG.HB.useHistogram = false; // only used during day } // ========== Done with dark frams / day / night settings + CG.myModeMeanSetting.minMean = CG.myModeMeanSetting.currentMean - CG.myModeMeanSetting.currentMean_threshold; + CG.myModeMeanSetting.maxMean = CG.myModeMeanSetting.currentMean + CG.myModeMeanSetting.currentMean_threshold; + + CG.myModeMeanSetting.minMean *= 255; // our algorithm compares to 0 - 255 + CG.myModeMeanSetting.maxMean *= 255; + + Log(3, "xxxxxxxxxxxxx minMean=%.3f, maxMean=%.3f\n", CG.myModeMeanSetting.minMean, CG.myModeMeanSetting.maxMean); + if (CG.myModeMeanSetting.currentMean > 0.0) { CG.myModeMeanSetting.modeMean = true; /* TODO: FUTURE - myModeMeanSetting.meanValue = CG.myModeMeanSetting.currentMean; if (! aegInit(cg, minExposure_us, CG.cameraMinGain, myRaspistillSetting, myModeMeanSetting)) { closeUp(EXIT_ERROR_STOP); @@ -1197,20 +1266,8 @@ int main(int argc, char *argv[]) } if (CG.isCooledCamera) { - asiRetCode = setControl(CG.cameraNumber, ASI_COOLER_ON, CG.currentEnableCooler ? ASI_TRUE : ASI_FALSE, ASI_FALSE); - if (asiRetCode != ASI_SUCCESS) - { - Log(1, "%s", c(KYEL)); - Log(1, " WARNING: Could not change cooler state: %s; continuing.\n", getRetCode(asiRetCode)); - Log(1, "%s", c(KNRM)); - } - asiRetCode = setControl(CG.cameraNumber, ASI_TARGET_TEMP, CG.currentTargetTemp, ASI_FALSE); - if (asiRetCode != ASI_SUCCESS) - { - Log(1, "%s", c(KYEL)); - Log(1, " WARNING: Could not set cooler temperature: %s; continuing.\n", getRetCode(asiRetCode)); - Log(1, "%s", c(KNRM)); - } + setControl(CG.cameraNumber, ASI_COOLER_ON, CG.currentEnableCooler ? ASI_TRUE : ASI_FALSE, ASI_FALSE); + setControl(CG.cameraNumber, ASI_TARGET_TEMP, CG.currentTargetTemp, ASI_FALSE); } setControl(CG.cameraNumber, ASI_GAIN, (long)CG.currentGain + gainChange, CG.currentAutoGain ? ASI_TRUE : ASI_FALSE); @@ -1223,11 +1280,6 @@ int main(int argc, char *argv[]) setControl(CG.cameraNumber, ASI_AUTO_TARGET_BRIGHTNESS, CG.currentBrightness, ASI_FALSE); } -#ifndef USE_HISTOGRAM - setControl(CG.cameraNumber, ASI_EXPOSURE, CG.currentExposure_us, CG.currentAutoExposure ? ASI_TRUE : ASI_FALSE); - // If not using histogram algorithm, ASI_EXPOSURE is set in takeOneExposure() -#endif - if (numExposures == 0 || CG.dayBin != CG.nightBin) { // Adjusting variables for chosen binning. @@ -1265,12 +1317,14 @@ int main(int argc, char *argv[]) { if (asiRetCode == ASI_ERROR_INVALID_SIZE) { - Log(0, "*** ERROR: your camera does not support bin %dx%d.\n", CG.currentBin, CG.currentBin); + Log(0, "*** %s: ERROR: your camera does not support bin %dx%d.\n", CG.ME, CG.currentBin, CG.currentBin); closeUp(EXIT_ERROR_STOP); } else { - Log(0, "*** ERROR: ASISetROIFormat(%d, %dx%d, %d, %d) failed (%s)\n", CG.cameraNumber, CG.width, CG.height, CG.currentBin, CG.imageType, getRetCode(asiRetCode)); + Log(0, "*** %s: ERROR: ASISetROIFormat(%d, %dx%d, %d, %d) failed (%s)\n", + CG.ME, CG.cameraNumber, CG.width, CG.height, CG.currentBin, + CG.imageType, getRetCode(asiRetCode)); closeUp(EXIT_ERROR_STOP); } } @@ -1279,12 +1333,7 @@ int main(int argc, char *argv[]) // Here and below, indent sub-messages with " > " so it's clear they go with the un-indented line. // This simply makes it easier to see things in the log file. -#ifdef USE_HISTOGRAM int attempts = 0; - int histogram[256]; -#else - int *histogram = NULL; -#endif // Wait for switch day time -> night time or night time -> day time while (bMain && lastDayOrNight == dayOrNight) @@ -1305,11 +1354,12 @@ int main(int argc, char *argv[]) sprintf(bufTime, "%s", formatTime(exposureStartDateTime, CG.timeFormat)); } - asiRetCode = takeOneExposure(&CG, pRgb.data, histogram); + asiRetCode = takeOneExposure(&CG, pRgb.data); if (asiRetCode == ASI_SUCCESS) { numErrors = 0; numExposures++; + bool hitMinOrMax = false; CG.lastFocusMetric = CG.overlay.showFocus ? (int)round(get_focus_metric(pRgb)) : -1; @@ -1320,24 +1370,19 @@ int main(int argc, char *argv[]) pthread_create(&threadDisplay, NULL, Display, (void *)&pRgb); } -#ifdef USE_HISTOGRAM - // We don't use this at night since the ZWO bug is only when it's light outside. if (CG.HB.useHistogram) { - attempts = 0; - - int minAcceptableMean = MINMEAN; - int maxAcceptableMean = MAXMEAN; -//xxx int roundToMe = 5; // round exposures to this many microseconds + // Make sure the mean is acceptable. - long newExposure_us = 0; + attempts = 0; - // histMinExposure_us is the min exposure used in the histogram calculation. -// xxx TODO: dump histMinExposure_us? Set tempMinExposure_us = cameraMinExposure_us ? ... - long histMinExposure_us = CG.cameraMinExposure_us; - long tempMinExposure_us = histMinExposure_us; + int minAcceptableMean = CG.myModeMeanSetting.minMean; + int maxAcceptableMean = CG.myModeMeanSetting.maxMean; + long tempMinExposure_us = CG.cameraMinExposure_us; long tempMaxExposure_us = CG.cameraMaxExposure_us; + long newExposure_us = 0; +// TODO: dump Brightness - user can adjust Target Mean or Manual Exposure. if (CG.currentBrightness != CG.defaultBrightness) { // Adjust brightness based on Brightness. @@ -1373,8 +1418,6 @@ int main(int argc, char *argv[]) } // Now adjust the variables -// xxxxxxxxx TODO: don't adjust histMinExposure_us; just histogram numbers. - histMinExposure_us *= exposureAdjustment; minAcceptableMean *= exposureAdjustment; maxAcceptableMean *= exposureAdjustment; } @@ -1384,106 +1427,84 @@ int main(int argc, char *argv[]) // When that happens we don't want to set the min to the second exposure // or else we'll never get low enough. // Negative is below lower limit, positive is above upper limit. - // Adjust the min or maxAcceptableMean depending on the aggression. - int priorMean = CG.lastMean; + int priorMean = NOT_SET; // The mean for the image before the last one. int priorMeanDiff = 0; - int adjustment = 0; - int lastMeanDiff = 0; // like priorMeanDiff but for next exposure + int numPingPongs = 0; if (CG.lastMean < minAcceptableMean) { priorMeanDiff = CG.lastMean - minAcceptableMean; - // If we're skipping frames we want to get to a good exposure as fast as - // possible so don't set an adjustment. -/* - if (CG.aggression != 100 && CG.currentSkipFrames <= 0) - { -// TODO: why are we adjusting the AcceptableMean? - adjustment = priorMeanDiff * (1 - ((float)CG.aggression/100)); - if (adjustment < 1) - minAcceptableMean += adjustment; - } -*/ } else if (CG.lastMean > maxAcceptableMean) { -// TODO: why not adjust here if needed? priorMeanDiff = CG.lastMean - maxAcceptableMean; } - if (adjustment != 0) - { - Log(4, " > !!! Adjusting %sAcceptableMean by %d to %d\n", - adjustment < 0 ? "min" : "max", - adjustment, - adjustment < 0 ? minAcceptableMean : maxAcceptableMean); - } - int numPingPongs = 0; -//x long lastExposure_us = CG.currentExposure_us; + // Keep trying until we get an acceptable mean or are unable to continue. while ((CG.lastMean < minAcceptableMean || CG.lastMean > maxAcceptableMean) && - ++attempts <= maxHistogramAttempts && CG.currentExposure_us <= CG.cameraMaxExposure_us) + ++attempts <= maxHistogramAttempts) { - int acceptable; + int acceptableMean; float multiplier = 1.10; char const *acceptableType; if (CG.lastMean < minAcceptableMean) { - acceptable = minAcceptableMean; + acceptableMean = minAcceptableMean; acceptableType = "min"; } else { - acceptable = maxAcceptableMean; + acceptableMean = maxAcceptableMean; acceptableType = "max"; multiplier = 1 / multiplier; } - // if lastMean/acceptable is 9/90, it's 1/10th of the way there, so multiple exposure by 90/9 (10). + // If lastMean/acceptableMean is 9/90, it's 1/10th of the way there, + // so multiple exposure by 90/9 (10). // ZWO cameras don't appear to be linear so increase the multiplier amount some. float multiply; if (CG.lastMean == 0) { // TODO: is this correct? - multiply = ((double)acceptable) * multiplier; + multiply = ((double)acceptableMean) * multiplier; } else { - multiply = ((double)acceptable / CG.lastMean) * multiplier; + multiply = ((double)acceptableMean / CG.lastMean) * multiplier; } -// Log(4, "multiply=%f, acceptable=%d, lastMean=%f, multiplier=%f\n", multiply, acceptable, CG.lastMean, multiplier); long exposureDiff_us = (CG.lastExposure_us * multiply) - CG.lastExposure_us; + long exposureDiffBeforeAgression_us = exposureDiff_us; // Adjust by aggression setting. - if (CG.aggression != 100 && CG.currentSkipFrames <= 0) + if (CG.aggression != 100 && CG.currentSkipFrames <= 0 && exposureDiff_us != 0) { - if (exposureDiff_us != 0) - { - Log(4, " > Next exposure change going from %s, ", length_in_units(exposureDiff_us, true)); - exposureDiff_us *= (float)CG.aggression / 100; - Log(4, "before aggression to %s after.\n", length_in_units(exposureDiff_us, true)); - } + exposureDiff_us *= (float)CG.aggression / 100; } + newExposure_us = CG.lastExposure_us + exposureDiff_us; + // Assume max auto exposure is <= max camera exposure. if (newExposure_us > CG.currentMaxAutoExposure_us) { - Log(4, " > === Calculated newExposure_us (%'ld) > currentMaxAutoExposure_us (%'ld); setting to max\n", newExposure_us, CG.currentMaxAutoExposure_us); + hitMinOrMax = true; + Log(3, " > === Calculated newExposure_us (%'ld) > CG.currentMaxAutoExposure_us (%'ld); setting to max\n", + newExposure_us, CG.currentMaxAutoExposure_us); newExposure_us = CG.currentMaxAutoExposure_us; } else { - Log(4, " > Next exposure changing by %'ld us to %'ld (multiply by %.3f) [CG.lastExposure_us=%'ld, %sAcceptable=%d, lastMean=%d]\n", - exposureDiff_us, newExposure_us, multiply, CG.lastExposure_us, acceptableType, acceptable, (int)CG.lastMean); + Log(3, " > Next exposure change: %'ld us (%'ld pre agression) to %'ld (* %.3f) [CG.lastExposure_us=%'ld, %sAcceptableMean=%d, CG.lastMean=%d]\n", + exposureDiff_us, exposureDiffBeforeAgression_us, + newExposure_us, multiply, CG.lastExposure_us, + acceptableType, acceptableMean, (int)CG.lastMean); } if (priorMeanDiff > 0 && lastMeanDiff < 0) { ++numPingPongs; - Log(2, " >xxx lastMean was %d and went from %d above max of %d to %d below min", - priorMean, priorMeanDiff, maxAcceptableMean, -lastMeanDiff); - Log(2, " of %d, is now at %d; should NOT set temp min to currentExposure_us of %'ld\n", - minAcceptableMean, (int)CG.lastMean, CG.currentExposure_us); + Log(2, " > xxx lastMean was %d and went from %d above max of %d to %d below min of %d, is now at %d;\n", + priorMean, priorMeanDiff, maxAcceptableMean, -lastMeanDiff, + minAcceptableMean, (int)CG.lastMean); } else { if (priorMeanDiff < 0 && lastMeanDiff > 0) { ++numPingPongs; - Log(2, " >xxx mean was %d and went from %d below min of %d to %d above max", - priorMean, -priorMeanDiff, minAcceptableMean, lastMeanDiff); - Log(2, " of %d, is now at %d; OK to set temp max to currentExposure_us of %'ld\n", - maxAcceptableMean, (int)CG.lastMean, CG.currentExposure_us); + Log(2, " > xxx lastMean was %d and went from %d below min of %d to %d above max of %d, is now at %d;\n", + priorMean, -priorMeanDiff, minAcceptableMean, lastMeanDiff, + maxAcceptableMean, (int)CG.lastMean); } else { @@ -1502,41 +1523,55 @@ int main(int argc, char *argv[]) if (numPingPongs >= 3) { -printf(" > xxx newExposure_us=%s\n", length_in_units(newExposure_us, true)); -printf(" CG.lastExposure_us=%s\n", length_in_units(CG.lastExposure_us, true)); +printf(" > xxx newExposure_us=%s, CG.lastExposure_us=%s, CG.currentExposure_us=%s,", + length_in_units(newExposure_us, true), length_in_units(CG.lastExposure_us, true), + length_in_units(CG.currentExposure_us, true)); newExposure_us = (newExposure_us + CG.lastExposure_us) / 2; -long n = newExposure_us; -printf(" new newExposure_us=%s\n", length_in_units(n, true)); +printf(" new newExposure_us=%s\n", length_in_units(newExposure_us, true)); Log(3, " > Ping-Ponged %d times, setting exposure to mid-point of %s\n", numPingPongs, length_in_units(newExposure_us, true)); + +// XXXX testing + // To try and help, add (or subtract) the numPingPongs percent to the exposure. + // For example, if newExposure_us == 200 and numPingPongs == 4, add 4% (8 us = 4% * 200). + long us = (long) (newExposure_us * ((double)numPingPongs / 100.0)); +Log(3, "================ Adding %'ld us\n", us); + newExposure_us += us; + if (tempMaxExposure_us < newExposure_us) + tempMaxExposure_us = newExposure_us; } -//xxx newExposure_us = roundTo(newExposure_us, roundToMe); // Make sure newExposure_us is between min and max. +// XXXX testing +long saved_newExposure_us = newExposure_us; newExposure_us = std::max(tempMinExposure_us, newExposure_us); newExposure_us = std::min(tempMaxExposure_us, newExposure_us); +if (saved_newExposure_us != newExposure_us) +{ + Log(3, "> xxx newExposure_us changed from %s to %s due to tempMin/tempMax\n", + length_in_units(saved_newExposure_us, true), length_in_units(newExposure_us, true)); +} if (newExposure_us == CG.currentExposure_us) { - break; + break; // message about this is output below } CG.currentExposure_us = newExposure_us; if (CG.currentExposure_us > CG.cameraMaxExposure_us) { - break; + hitMinOrMax = true; + break; // message about this is output below } - Log(2, " >> Retry %i @ %'ld us, min=%'ld us, max=%'ld us: lastMean (%d)\n", - attempts, newExposure_us, tempMinExposure_us, tempMaxExposure_us, (int)CG.lastMean); + Log(2, " >> Retry %i @ %'ld us, min=%'ld us, max=%'ld us\n", + attempts, newExposure_us, tempMinExposure_us, tempMaxExposure_us); priorMean = CG.lastMean; priorMeanDiff = lastMeanDiff; - asiRetCode = takeOneExposure(&CG, pRgb.data, histogram); + asiRetCode = takeOneExposure(&CG, pRgb.data); if (asiRetCode == ASI_SUCCESS) { -//x lastExposure_us = CG.lastExposure_us; - if (CG.lastMean < minAcceptableMean) lastMeanDiff = CG.lastMean - minAcceptableMean; else if (CG.lastMean > maxAcceptableMean) @@ -1554,7 +1589,8 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); if (asiRetCode != ASI_SUCCESS) { - Log(2," > Sleeping %s from failed exposure\n", length_in_units(CG.currentDelay_ms * US_IN_MS, false)); + Log(2," > Sleeping %s from failed exposure\n", + length_in_units(CG.currentDelay_ms * US_IN_MS, false)); usleep(CG.currentDelay_ms * US_IN_MS); // Don't save the file or do anything below. continue; @@ -1563,36 +1599,45 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); if (CG.lastMean >= minAcceptableMean && CG.lastMean <= maxAcceptableMean) { // +++ at end makes it easier to see in log file - Log(2, " > Good image: mean within range of %d to %d ++++++++++, mean %d\n", minAcceptableMean, maxAcceptableMean, (int)CG.lastMean); + Log(2, " > Good image: mean within range of %d to %d ++++++++++, mean %d\n", + minAcceptableMean, maxAcceptableMean, (int)CG.lastMean); } else if (attempts > maxHistogramAttempts) { - Log(2, " > max attempts reached - using exposure of %s with mean %d\n", length_in_units(CG.currentExposure_us, true), (int)CG.lastMean); + Log(2, " > max attempts reached - using exposure of %s with mean %d\n", + length_in_units(CG.currentExposure_us, true), (int)CG.lastMean); } else if (attempts >= 1) { if (CG.currentExposure_us < CG.cameraMinExposure_us) { - // If we call length_in_units() twice in same command line they both return the last value. - Log(2, " > Stopped trying: new exposure of %s ", length_in_units(CG.currentExposure_us, false)); - Log(2, "would be over min of %s\n", length_in_units(CG.cameraMinExposure_us, false)); + hitMinOrMax = true; + Log(2, " > Stopped trying: new exposure of %s would be under camera min of %s\n", + length_in_units(CG.currentExposure_us, false), + length_in_units(CG.cameraMinExposure_us, false)); long diff = (long)((float)CG.currentExposure_us * (1/(float)percentChange)); CG.currentExposure_us += diff; - Log(3, " > Increasing next exposure by %d%% (%'ld us) to %'ld\n", percentChange, diff, CG.currentExposure_us); + Log(3, " > Increasing next exposure by %d%% (%'ld us) to %'ld\n", + percentChange, diff, CG.currentExposure_us); } else if (CG.currentExposure_us > CG.cameraMaxExposure_us) { - Log(2, " > Stopped trying: new exposure of %s ", length_in_units(CG.currentExposure_us, false)); - Log(2, "would be over max of %s\n", length_in_units(CG.cameraMaxExposure_us, false)); + hitMinOrMax = true; + Log(2, " > Stopped trying: new exposure of %s would be over camera max of %s\n", + length_in_units(CG.currentExposure_us, false), + length_in_units(CG.cameraMaxExposure_us, false)); long diff = (long)((float)CG.currentExposure_us * (1/(float)percentChange)); CG.currentExposure_us -= diff; - Log(3, " > Decreasing next exposure by %d%% (%'ld us) to %'ld\n", percentChange, diff, CG.currentExposure_us); + Log(3, " > Decreasing next exposure by %d%% (%'ld us) to %'ld\n", + percentChange, diff, CG.currentExposure_us); } else if (CG.currentExposure_us == CG.cameraMinExposure_us) { - Log(2, " > Stopped trying: hit min exposure limit of %s, mean %d\n", length_in_units(CG.cameraMinExposure_us, false), (int)CG.lastMean); + Log(2, " > Stopped trying: hit camera min exposure limit of %s\n", + length_in_units(CG.cameraMinExposure_us, false)); + // If currentExposure_us causes too low of a mean, increase exposure // so on the next loop we'll adjust it. if (CG.lastMean < minAcceptableMean) @@ -1600,7 +1645,8 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); } else if (CG.currentExposure_us == CG.currentMaxAutoExposure_us) { - Log(2, " > Stopped trying: hit max exposure limit of %s, mean %d\n", length_in_units(CG.currentMaxAutoExposure_us, false), (int)CG.lastMean); + Log(2, " > Stopped trying: hit max autoexposure limit of %s\n", + length_in_units(CG.currentMaxAutoExposure_us, false)); // If currentExposure_us causes too high of a mean, decrease exposure // so on the next loop we'll adjust it. if (CG.lastMean > maxAcceptableMean) @@ -1614,7 +1660,8 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); else { Log(2, " > Stopped trying, using exposure of %s with mean %d, min=%d, max=%d\n", - length_in_units(CG.currentExposure_us, false), (int)CG.lastMean, minAcceptableMean, maxAcceptableMean); + length_in_units(CG.currentExposure_us, false), + (int)CG.lastMean, minAcceptableMean, maxAcceptableMean); } } @@ -1645,11 +1692,12 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); exposureDiff_us = diff_us * (float)CG.aggression / 100; if (exposureDiff_us != 0) { - Log(4, " > Next exposure full change is %s, ", length_in_units(diff_us, true)); - Log(4, "after aggression: %s ", length_in_units(exposureDiff_us, true)); - Log(4, "from %s ", length_in_units(CG.currentExposure_us, true)); + Log(3, " > Next exposure full change is %s, after agression: %s from %s ", + length_in_units(diff_us, true), + length_in_units(exposureDiff_us, true), + length_in_units(CG.currentExposure_us, true)); CG.currentExposure_us += exposureDiff_us; - Log(4, "to %s\n", length_in_units(CG.currentExposure_us, true)); + Log(3, "to %s\n", length_in_units(CG.currentExposure_us, true)); } } else @@ -1659,23 +1707,19 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); } else { - // Didn't use auto-exposure - don't change exposure + // Using manual exposure so don't change exposure. } } -#endif + if (CG.currentSkipFrames > 0) { -#ifdef USE_HISTOGRAM - // If we're already at a good exposure, or the last exposure was longer - // than the max, don't skip any more frames. -// xxx TODO: should we have a separate variable to define "too long" instead of currentMaxAutoExposure_us? - if ((CG.lastMean >= MINMEAN && CG.lastMean <= MAXMEAN) || CG.lastExposure_us > CG.currentMaxAutoExposure_us) + // If we're already at a good exposure, or the last exposure reached the max or min time, + // don't skip any more frames. + if ((CG.lastMean >= CG.myModeMeanSetting.minMean && CG.lastMean <= CG.myModeMeanSetting.maxMean) + || hitMinOrMax) { CG.currentSkipFrames = 0; - } - else -#endif - { + } else { CG.currentSkipFrames--; Log(2, " >>>> Skipping this frame. %d left to skip\n", CG.currentSkipFrames); // Do not save this frame or sleep after it. @@ -1690,8 +1734,6 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); if (CG.overlay.overlayMethod == OVERLAY_METHOD_LEGACY) { (void) doOverlay(pRgb, CG, bufTime, gainChange); - -#ifdef USE_HISTOGRAM if (CG.overlay.showHistogramBox) { // Draw a rectangle where the histogram box is. @@ -1708,7 +1750,6 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); cv::rectangle(pRgb, cv::Point(X1, Y1), cv::Point(X2, Y2), outerLine, thickness, lt, 0); cv::rectangle(pRgb, cv::Point(X1+thickness, Y1+thickness), cv::Point(X2-thickness, Y2-thickness), innerLine, thickness, lt, 0); } -#endif } if (currentAdjustGain) { @@ -1719,15 +1760,6 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); } } -#ifndef USE_HISTOGRAM - if (CG.currentAutoExposure) - { - // Retrieve the current Exposure for smooth transition to night time - // as long as auto-exposure is enabled during night time - CG.currentExposure_us = CG.lastExposure_us; - } -#endif - // Save the image if (! bSavingImg) { @@ -1754,25 +1786,16 @@ printf(" new newExposure_us=%s\n", length_in_units(n, true)); // TODO: wait for the prior image to finish saving. } -#ifndef USE_HISTOGRAM - - if (CG.currentAutoExposure && dayOrNight == "DAY") - { - CG.currentExposure_us = CG.lastExposure_us; - } -#endif std::string s; if (CG.currentAutoExposure) { s = "auto"; } - else + else if (CG.HB.useHistogram) { + s = "histogram"; + } else { s = "manual"; -#ifdef USE_HISTOGRAM - if (CG.HB.useHistogram) - s = "histogram"; -#endif } // Delay applied before next exposure delayBetweenImages(CG, CG.lastExposure_us, s); diff --git a/src/include/allsky_common.h b/src/include/allsky_common.h index 1dadd46d1..15135da05 100644 --- a/src/include/allsky_common.h +++ b/src/include/allsky_common.h @@ -36,8 +36,35 @@ // Defaults #define NO_MAX_VALUE 9999999 // signifies a number has no maximum value #define AUTO_IMAGE_TYPE 99 // must match what's in the camera_settings.json file -#define DEFAULT_DAYMEAN 0.5 -#define DEFAULT_NIGHTMEAN 0.2 + +#define DEFAULT_DAYMEAN_RPi 0.5 // target value +#define DEFAULT_DAYMEAN_THRESHOLD_RPi 0.1 // mean brightness must be within this % to be "ok" +#define DEFAULT_NIGHTMEAN_RPi 0.2 // target value +#define DEFAULT_NIGHTMEAN_THRESHOLD_RPi 0.1 // target value +#define DEFAULT_MEAN_P0_RPi 5.0 +#define DEFAULT_MEAN_P1_RPi 20.0 +#define DEFAULT_MEAN_P2_RPi 45.0 +#define DEFAULT_MINMEAN_P_RPi 0.0 +#define DEFAULT_MAXMEAN_P_RPi 50.0 +#define DEFAULT_MINMEAN_RPi 0.0 +#define DEFAULT_MAXMEAN_RPi 1.0 +#define DEFAULT_MINMEAN_THRESHOLD_RPi 0.0 +#define DEFAULT_MAXMEAN_THRESHOLD_RPi 1.0 + +// Got these by trial and error. 128 is more-or-less half the max of 255. +#define DEFAULT_DAYMEAN_ZWO (128.0 / 255) // matches old way +#define DEFAULT_DAYMEAN_THRESHOLD_ZWO (6.0 / 255) // matches old way +#define DEFAULT_NIGHTMEAN_ZWO (75.0 / 255) // TODO: pure guess as of May 22, 2023 +#define DEFAULT_NIGHTMEAN_THRESHOLD_ZWO (6.0 / 255) +#define DEFAULT_MEAN_P0_ZWO 5.0 // TODO: set after porting modemean to ZWO +#define DEFAULT_MEAN_P1_ZWO 20.0 // TODO: set after porting modemean to ZWO +#define DEFAULT_MEAN_P2_ZWO 45.0 // TODO: set after porting modemean to ZWO +#define DEFAULT_MINMEAN_P_ZWO 0.0 // TODO: set after porting modemean to ZWO +#define DEFAULT_MAXMEAN_P_ZWO 50.0 // TODO: set after porting modemean to ZWO +#define DEFAULT_MINMEAN_ZWO DEFAULT_MINMEAN_RPi +#define DEFAULT_MAXMEAN_ZWO DEFAULT_MAXMEAN_RPi +#define DEFAULT_MINMEAN_THRESHOLD_ZWO DEFAULT_MINMEAN_THRESHOLD_RPi +#define DEFAULT_MAXMEAN_THRESHOLD_ZWO DEFAULT_MAXMEAN_THRESHOLD_RPi // Default overlay values - will go away once external overlay program is implemented #define SMALLFONTSIZE_MULTIPLIER 0.08 @@ -100,6 +127,7 @@ struct overlay { // Histogram Box, ZWO only struct HB { bool useHistogram = false; // Should we use histogram auto-exposure? + bool useExperimentalExposure = false; // Should histogram auto-exposure at night? int histogramBoxSizeX = 500; // width of box in pixels int currentHistogramBoxSizeX = NOT_CHANGED; int histogramBoxSizeY = 500; // height of box in pixels @@ -117,14 +145,27 @@ struct HB { struct myModeMeanSetting { bool modeMean = false; // currently using it? - double dayMean = DEFAULT_DAYMEAN; - double nightMean = DEFAULT_NIGHTMEAN; - double currentMean = NOT_SET; // (calculated value) - double Mean = NOT_SET; // (calculated value) - double mean_threshold = 0.1; - double mean_p0 = 5.0; - double mean_p1 = 20.0; - double mean_p2 = 45.0; + double dayMean = NOT_SET; // initialized at runtime + double nightMean = NOT_SET; // initialized at runtime + double currentMean = NOT_SET; // holds either day or night mean + + double Mean = NOT_SET; // calculated value after exposure + double minMean = NOT_SET; // initialized at runtime + double maxMean = NOT_SET; // initialized at runtime + + double dayMean_threshold = NOT_SET; // initialized at runtime + double nightMean_threshold = NOT_SET; // initialized at runtime + double currentMean_threshold = NOT_SET; // holds either day or night threshold + double minMean_threshold = NOT_SET; // initialized at runtime + double maxMean_threshold = NOT_SET; // initialized at runtime +// TODO: only use day and night versions xxxxxxxx will be deleted + double mean_threshold = NOT_SET; // initialized at runtime + // ExposureChange (Steps) = p0 + p1 * diff + (p2*diff)^2 + double mean_p0 = NOT_SET; // initialized at runtime + double mean_p1 = NOT_SET; // initialized at runtime + double mean_p2 = NOT_SET; // initialized at runtime + double minMean_p = NOT_SET; // initialized at runtime + double maxMean_p = NOT_SET; // initialized at runtime }; @@ -302,6 +343,7 @@ struct config { // for configuration variables long lastFocusMetric = NOT_SET; long lastAsiBandwidth = NOT_SET; double lastMean = NOT_SET; + double lastMeanFull = NOT_SET; bool goodLastExposure = false; // Was the last image propery exposed? }; diff --git a/src/include/mode_mean.h b/src/include/mode_mean.h index daf022e08..275a86c1d 100644 --- a/src/include/mode_mean.h +++ b/src/include/mode_mean.h @@ -1,9 +1,21 @@ #pragma once -#define DEFAULT_MEAN_P0 5.0 -#define DEFAULT_MEAN_P1 20.0 -#define DEFAULT_MEAN_P2 45.0 -#define DEFAULT_MEAN_THRESHOLD 0.1 // mean brightness must be within this percent to be "ok" +// https://www.raspberrypi.org/documentation/raspbian/applications/camera.md +struct raspistillSetting { + // Sets the analog gain value directly on the sensor. + double analoggain = 1.0; + + // Sets the digital gain value applied by the ISP (floating point value + // from 1.0 to 64.0, but values over about 4.0 will produce overexposed images). + double digitalgain = 1.0; + + // Sets the shutter open time to the specified value (in microseconds). + int shutter_us = 1*US_IN_SEC; + + // Sets the brightness of the image. 50 is the default. 0 is black, 100 is white. + int brightness = 50; +}; + typedef enum { // defined exposure and gain is used @@ -20,29 +32,28 @@ typedef enum { } MEAN_AUTO_MODE; struct modeMeanSetting { - bool modeMean = false; // Activate mode mean. User can change this. - MEAN_AUTO_MODE meanAuto = MEAN_AUTO_OFF; // Different modes are available - see MEAN_AUTO_MODE. - bool init = true; // Set some settings before first calculation. + bool modeMean = false; // Use mode mean algorithm? User can change this. + MEAN_AUTO_MODE meanAuto = MEAN_AUTO_OFF; // Different modes are available. + bool init = true; // Set some settings before first calculation. // This is set to "false" after calculation. - int exposureLevelMin = NOT_SET; // Set during initialization. - int exposureLevelMax = NOT_SET; // Set during initialization. - int exposureLevel = NOT_SET; // current ExposureLevel. - double minGain = NOT_SET; // Set during initialization. - double maxGain = NOT_SET; // Set during initialization. - long maxExposure_us = NOT_SET; // Set during initialization. - long minExposure_us = NOT_SET; // Set during initialization. - double meanValue = NOT_SET; // Default mean value for well exposed images. + int exposureLevelMin = NOT_SET; // Set during initialization. + int exposureLevelMax = NOT_SET; // Set during initialization. + int exposureLevel = NOT_SET; // current ExposureLevel. + double minGain = NOT_SET; // Set during initialization. + double maxGain = NOT_SET; // Set during initialization. + long maxExposure_us = NOT_SET; // Set during initialization. + long minExposure_us = NOT_SET; // Set during initialization. // Default mean value for daytime and nighttime images. User can change. - double dayMean = DEFAULT_DAYMEAN; - double nightMean = DEFAULT_NIGHTMEAN; - double mean_threshold = DEFAULT_MEAN_THRESHOLD; // threshold value. User can change. - - double const shuttersteps = 6.0; // shuttersteps - int const historySize = 3; // Number of last images for mean target calculation. - double mean_p0 = DEFAULT_MEAN_P0; // ExposureChange (Steps) = p0 + p1 * diff + (p2*diff)^2 - double mean_p1 = DEFAULT_MEAN_P1; - double mean_p2 = DEFAULT_MEAN_P2; + double dayMean = DEFAULT_DAYMEAN_RPi; + double nightMean = DEFAULT_NIGHTMEAN_RPi; + + // Default threshold values for daytime and nighttime. User can change. + double dayMean_threshold = DEFAULT_DAYMEAN_THRESHOLD_RPi; + double nightMean_threshold = DEFAULT_NIGHTMEAN_THRESHOLD_RPi; + + double const shuttersteps = 6.0; // shuttersteps + int const historySize = 3; // Number of last images for mean target calculation. }; bool aegInit(config, raspistillSetting &, modeMeanSetting &); diff --git a/src/include/raspistill.h b/src/include/raspistill.h deleted file mode 100644 index de09de4d5..000000000 --- a/src/include/raspistill.h +++ /dev/null @@ -1,11 +0,0 @@ -// https://www.raspberrypi.org/documentation/raspbian/applications/camera.md - -#pragma once - -struct raspistillSetting { - double analoggain = 1.0; // Sets the analog gain value directly on the sensor. - double digitalgain = 1.0; // Sets the digital gain value applied by the ISP (floating point value - // from 1.0 to 64.0, but values over about 4.0 will produce overexposed images). - int shutter_us = 1*US_IN_SEC; // Sets the shutter open time to the specified value (in microseconds). - int brightness = 50; // Sets the brightness of the image. 50 is the default. 0 is black, 100 is white. -}; diff --git a/src/mode_mean.cpp b/src/mode_mean.cpp index 584d5f7ec..be6a9a001 100644 --- a/src/mode_mean.cpp +++ b/src/mode_mean.cpp @@ -17,8 +17,6 @@ #include #include "include/allsky_common.h" - -#include "include/raspistill.h" #include "include/mode_mean.h" // These only need to be as large as modeMeanSetting.historySize. @@ -60,14 +58,15 @@ bool aegInit(config cg, // XXXXXX Does this need to be done every transition between day and night, // or just once when Allsky starts? - // first exposure with currentRaspistillSetting.shutter_us, so we have to calculate the startpoint for ExposureLevel + // first exposure with currentRaspistillSetting.shutter_us, + // so we have to calculate the startpoint for ExposureLevel initialExposureLevel = calcExposureLevel(cg.currentExposure_us, cg.currentGain, currentModeMeanSetting) - 1; currentModeMeanSetting.exposureLevel = initialExposureLevel; currentRaspistillSetting.shutter_us = cg.currentExposure_us; for (int i=0; i < currentModeMeanSetting.historySize; i++) { // Pretend like all prior images had the target mean and initial exposure level. - meanHistory[i] = currentModeMeanSetting.meanValue; + meanHistory[i] = cg.myModeMeanSetting.currentMean; exposureLevelHistory[i] = initialExposureLevel; } } @@ -201,7 +200,7 @@ void aegGetNextExposureSettings(config * cg, Log(3, " > Got: shutter_us: %s, gain: %1.3f, mean: %1.3f, target mean: %1.3f, diff (target - mean): %'1.3f\n", length_in_units(currentRaspistillSetting.shutter_us, true), cg->lastGain, cg->lastMean, - currentModeMeanSetting.meanValue, (currentModeMeanSetting.meanValue - cg->lastMean)); + cg->myModeMeanSetting.currentMean, (cg->myModeMeanSetting.currentMean - cg->lastMean)); meanHistory[MeanCnt % currentModeMeanSetting.historySize] = cg->lastMean; @@ -233,35 +232,35 @@ void aegGetNextExposureSettings(config * cg, // same value as current value newMean += mean_forecast * currentModeMeanSetting.historySize; newMean /= (double) values; - mean_diff = abs(newMean - currentModeMeanSetting.meanValue); + mean_diff = abs(newMean - cg->myModeMeanSetting.currentMean); Log(3, " > New mean target: %1.3f, mean_forecast: %1.3f, mean_diff (newMean - target mean): %'1.3f, idx=%d, idxN1=%d\n", newMean, mean_forecast, mean_diff, idx, idxN1); int ExposureChange; - double const multiplier1 = 1.5; // xxxx was 2.0 + double const multiplier1 = 1.75; // xxxx was 2.0 double const multiplier2 = 1.25; - double meanDiff = abs(cg->lastMean - currentModeMeanSetting.meanValue); // xxx was = mean_diff + double meanDiff = abs(cg->lastMean - cg->myModeMeanSetting.currentMean); // xxx was = mean_diff // fast forward - if (fastforward || meanDiff > (currentModeMeanSetting.mean_threshold * multiplier1)) { + if (fastforward || meanDiff > (cg->myModeMeanSetting.currentMean_threshold * multiplier1)) { // We are fairly far off from desired mean so make a big change next time. - ExposureChange = std::max(1.0, currentModeMeanSetting.mean_p0 + (currentModeMeanSetting.mean_p1 * mean_diff) + pow(currentModeMeanSetting.mean_p2 * mean_diff, 2.0)); + ExposureChange = std::max(1.0, cg->myModeMeanSetting.mean_p0 + (cg->myModeMeanSetting.mean_p1 * mean_diff) + pow(cg->myModeMeanSetting.mean_p2 * mean_diff, 2.0)); Log(3, " > fast forward ExposureChange now %d (meanDiff=%1.3f > %.2f*threshold=%1.3f)\n", - ExposureChange, meanDiff, multiplier1, currentModeMeanSetting.mean_threshold*multiplier1); + ExposureChange, meanDiff, multiplier1, cg->myModeMeanSetting.currentMean_threshold * multiplier1); } - else if (meanDiff > (currentModeMeanSetting.mean_threshold * multiplier2)) { + else if (meanDiff > (cg->myModeMeanSetting.currentMean_threshold * multiplier2)) { // We are somewhat far off from desired mean so make a big change next time. - ExposureChange = std::max(1.0, currentModeMeanSetting.mean_p0 + (currentModeMeanSetting.mean_p1 * mean_diff) + (pow(currentModeMeanSetting.mean_p2 * mean_diff, 2.0) / 2.0)); + ExposureChange = std::max(1.0, cg->myModeMeanSetting.mean_p0 + (cg->myModeMeanSetting.mean_p1 * mean_diff) + (pow(cg->myModeMeanSetting.mean_p2 * mean_diff, 2.0) / 2.0)); Log(3, " > medium forward ExposureChange now %d (meanDiff=%1.3f > %.2f*threshold=%1.3f)\n", - ExposureChange, meanDiff, multiplier2, currentModeMeanSetting.mean_threshold*multiplier2); + ExposureChange, meanDiff, multiplier2, cg->myModeMeanSetting.currentMean_threshold * multiplier2); } // slow forward - else if (meanDiff > currentModeMeanSetting.mean_threshold) { + else if (meanDiff > cg->myModeMeanSetting.currentMean_threshold) { // We are fairly close to desired mean so make a small change next time. - ExposureChange = std::max(1.0, currentModeMeanSetting.mean_p0 + currentModeMeanSetting.mean_p1 * mean_diff); + ExposureChange = std::max(1.0, cg->myModeMeanSetting.mean_p0 + cg->myModeMeanSetting.mean_p1 * mean_diff); Log(3, " > slow forward ExposureChange now %d (meanDiff=%1.3f, %.2f*threshold=%1.3f)\n", - ExposureChange, meanDiff, multiplier2, currentModeMeanSetting.mean_threshold*multiplier2); + ExposureChange, meanDiff, multiplier2, cg->myModeMeanSetting.currentMean_threshold * multiplier2); } else { // We are within the threshold @@ -276,9 +275,9 @@ void aegGetNextExposureSettings(config * cg, Log(4, " > ExposureChange clipped to %d (diff from last change: %d)\n", ExposureChange, dExposureChange); // If the last image's mean was good, no changes are needed to the next one. -// TODO: make mean_threshold a percent instead of an actual value. This will allow us to use 0 to 100 for what user enters as mean. +// TODO: make currentMean_threshold a percent instead of an actual value. This will allow us to use 0 to 100 for what user enters as mean. - if (cg->lastMean < (currentModeMeanSetting.meanValue - currentModeMeanSetting.mean_threshold)) { + if (cg->lastMean < (cg->myModeMeanSetting.currentMean - cg->myModeMeanSetting.currentMean_threshold)) { // mean too low if ((currentRaspistillSetting.analoggain < currentModeMeanSetting.maxGain) || (currentRaspistillSetting.shutter_us < currentModeMeanSetting.maxExposure_us)) { @@ -292,7 +291,7 @@ void aegGetNextExposureSettings(config * cg, currentModeMeanSetting.maxGain, length_in_units(currentModeMeanSetting.maxExposure_us, true)); } } - else if (cg->lastMean > (currentModeMeanSetting.meanValue + currentModeMeanSetting.mean_threshold)) { + else if (cg->lastMean > (cg->myModeMeanSetting.currentMean + cg->myModeMeanSetting.currentMean_threshold)) { // mean too high if ((currentRaspistillSetting.analoggain > currentModeMeanSetting.minGain) || (lastExposureTime_us > currentModeMeanSetting.minExposure_us)) { @@ -308,7 +307,7 @@ void aegGetNextExposureSettings(config * cg, } else { Log(3, " > ++++++++++ Prior image mean good - no changes needed, mean=%1.3f, target mean=%1.3f threshold=%1.3f\n", - cg->lastMean, currentModeMeanSetting.meanValue, currentModeMeanSetting.mean_threshold); + cg->lastMean, cg->myModeMeanSetting.currentMean, cg->myModeMeanSetting.currentMean_threshold); cg->goodLastExposure = true; } @@ -324,8 +323,8 @@ void aegGetNextExposureSettings(config * cg, Log(4, " > FF activated\n"); } if (fastforward && - (abs(meanHistory[idx] - currentModeMeanSetting.meanValue) < currentModeMeanSetting.mean_threshold) && - (abs(meanHistory[idxN1] - currentModeMeanSetting.meanValue) < currentModeMeanSetting.mean_threshold)) { + (abs(meanHistory[idx] - cg->myModeMeanSetting.currentMean) < cg->myModeMeanSetting.currentMean_threshold) && + (abs(meanHistory[idxN1] - cg->myModeMeanSetting.currentMean) < cg->myModeMeanSetting.currentMean_threshold)) { fastforward = false; Log(4, " > FF deactivated\n"); } diff --git a/src/sunwait-src b/src/sunwait-src index 102cb417e..151d8340a 160000 --- a/src/sunwait-src +++ b/src/sunwait-src @@ -1 +1 @@ -Subproject commit 102cb417ecbb7a3757ba9ee4b94d6db3225124c4 +Subproject commit 151d8340a748a4dac7752ebcd38983b2887f5f0c diff --git a/src/sunwait.patch b/src/sunwait.patch deleted file mode 100644 index b983bff3f..000000000 --- a/src/sunwait.patch +++ /dev/null @@ -1,34 +0,0 @@ -diff --git a/sunriset.cpp b/sunriset.cpp -index ed6ee0b..f7f7974 100755 ---- a/sunriset.cpp -+++ b/sunriset.cpp -@@ -148,7 +148,7 @@ void sunpos (const double d, double *lon, double *r) - void sun_RA_dec (const double d, double *RA, double *dec, double *r) - { - double lon, obl_ecl; -- double xs, ys, zs; -+ double xs, ys; - double xe, ye, ze; - - /* Compute Sun's ecliptical coordinates */ -@@ -157,7 +157,6 @@ void sun_RA_dec (const double d, double *RA, double *dec, double *r) - /* Compute ecliptic rectangular coordinates */ - xs = *r * cosd(lon); - ys = *r * sind(lon); -- zs = 0; /* because the Sun is always in the ecliptic plane! */ - - /* Compute obliquity of ecliptic (inclination of Earth's axis) */ - obl_ecl = 23.4393 - 3.563E-7 * d; -diff --git a/sunwait.cpp b/sunwait.cpp -index 7e43bc9..8ff6429 100755 ---- a/sunwait.cpp -+++ b/sunwait.cpp -@@ -662,7 +662,7 @@ int main (int argc, char *argv[]) - if (pRun->debug == ONOFF_ON) printf ("Debug: argv[%d]: >%s<\n", i, arg); - - // Strip any hyphen from arguments, but not negative signs of numbers -- if (arg[0] == '-' && arg[1] != '\0' && !isdigit(arg[1])) *arg++; -+ if (arg[0] == '-' && arg[1] != '\0' && !isdigit(arg[1])) memmove(arg, arg + 1, sizeof arg - 1); - - // Normal help or version info - if (!strcmp (arg, "v") || diff --git a/upgrade.sh b/upgrade.sh index a6d2efa21..0c507ad94 100755 --- a/upgrade.sh +++ b/upgrade.sh @@ -112,7 +112,7 @@ done [[ ${DEBUG} == "true" ]] && echo "Running: ${ME} ${ALL_ARGS}" -BRANCH="$( get_branch "${ALLSKY_BRANCH_FILE}" )" +BRANCH="$( get_branch )" [[ -z ${BRANCH} ]] && BRANCH="${GITHUB_MAIN_BRANCH}" # Unless forced to, only do the version check if we're on the main branch, diff --git a/variables.sh b/variables.sh index f5bfc0387..9c632eed0 100644 --- a/variables.sh +++ b/variables.sh @@ -20,7 +20,8 @@ if [[ -z "${ALLSKY_VARIABLE_SET}" ]]; then GREEN="\033[0;32m"; wOK="${GREEN}" YELLOW="\033[0;33m"; wWARNING="${YELLOW}" RED="\033[0;31m"; wERROR="${RED}" - DEBUG="${YELLOW}"; wDEBUG="${YELLOW}" + # Can't use DEBUG since lots of scripts use that to enable debugging + cDEBUG="${YELLOW}"; wDEBUG="${YELLOW}" NC="\033[0m"; wNC="${NC}" wBOLD="["; wNBOLD="]" wBR="\n" @@ -29,7 +30,7 @@ if [[ -z "${ALLSKY_VARIABLE_SET}" ]]; then GREEN=""; wOK="" YELLOW=""; wWARNING="" RED=""; wERROR="" - DEBUG=""; wDEBUG="${wWARNING}" + cDEBUG=""; wDEBUG="${wWARNING}" NC=""; wNC="" wBOLD=""; wNBOLD="" wBR="
    " @@ -74,13 +75,18 @@ if [[ -z "${ALLSKY_VARIABLE_SET}" ]]; then # Holds temporary messages to display in the WebUI. ALLSKY_MESSAGES="${ALLSKY_CONFIG}/messages.txt" + # Holds a count of continuous "bad" images + ALLSKY_BAD_IMAGE_COUNT="${ALLSKY_TMP}/bad_image_count.txt" + # Holds information on what the user needs to do after an installation. ALLSKY_INSTALLATION_LOGS="${ALLSKY_CONFIG}/installation_logs" POST_INSTALLATION_ACTIONS="${ALLSKY_INSTALLATION_LOGS}/post-installation_actions.txt" - # Holds temporary list of aborted uploads and timelapse since another one was in progress - ALLSKY_ABORTEDUPLOADS="${ALLSKY_TMP}/aborted_uploads.txt" - ALLSKY_ABORTEDTIMELAPSE="${ALLSKY_TMP}/aborted_timelapse.txt" + # Holds temporary list of aborted processes since another one was in progress. + ALLSKY_ABORTS_DIR="${ALLSKY_TMP}/aborts" + ALLSKY_ABORTEDUPLOADS="uploads.txt" + ALLSKY_ABORTEDTIMELAPSE="timelapse.txt" + ALLSKY_ABORTEDSAVEIMAGE="saveImage.txt" # Holds all the dark frames. ALLSKY_DARKS="${ALLSKY_HOME}/darks" @@ -94,9 +100,13 @@ if [[ -z "${ALLSKY_VARIABLE_SET}" ]]; then ALLSKY_MODULE_LOCATION="/opt/allsky" ALLSKY_EXTRA="${ALLSKY_OVERLAY}/extra" - # Verion file and option branch file. + # Directories and files for the flow timer function + ALLSKY_FLOWTIMINGS="${ALLSKY_TMP}/flowtimings" + ALLSKY_FLOWTIMINGS_DAY="${ALLSKY_FLOWTIMINGS}/day-average" + ALLSKY_FLOWTIMINGS_NIGHT="${ALLSKY_FLOWTIMINGS}/night-average" + + # Verion file. ALLSKY_VERSION_FILE="${ALLSKY_HOME}/version" - ALLSKY_BRANCH_FILE="${ALLSKY_HOME}/branch" # Location of optional allsky-website package. ALLSKY_WEBSITE="${ALLSKY_WEBUI}/allsky" @@ -112,6 +122,10 @@ if [[ -z "${ALLSKY_VARIABLE_SET}" ]]; then # Holds all the Allsky documentation. ALLSKY_DOCUMENTATION="${ALLSKY_WEBUI}/documentation" + # When the Pi was last rebooted. If the file exists a reboot is needed. + # Put in ALLSKY_TMP so it'll be removed upon reboot. + ALLSKY_REBOOT_NEEDED="${ALLSKY_TMP}/reboot_needed.txt" + # Log files for main Allsky and modules ALLSKY_LOG="/var/log/allsky.log" ALLSKY_PERIODIC_LOG="/var/log/allskyperiodic.log" diff --git a/version b/version index ebe520cec..6a29e506a 100644 --- a/version +++ b/version @@ -1 +1 @@ -v2023.05.01 +v2023.05.01_01 diff --git a/website/install.sh b/website/install.sh index 4bdf00289..72fbdcf83 100755 --- a/website/install.sh +++ b/website/install.sh @@ -204,7 +204,7 @@ get_versions_and_branches() fi if [[ ${PRIOR_WEBSITE_TYPE} == "new" ]]; then - PRIOR_WEBSITE_BRANCH="$( get_branch "${PRIOR_WEBSITE}/" )" + PRIOR_WEBSITE_BRANCH="$( get_branch "${PRIOR_WEBSITE}" )" fi PRIOR_WEBSITE_BRANCH="${PRIOR_WEBSITE_BRANCH:-${GITHUB_MAIN_BRANCH}}"