Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requested changes to flow timer #2717

Closed
EricClaeys opened this issue May 30, 2023 · 2 comments
Closed

Requested changes to flow timer #2717

EricClaeys opened this issue May 30, 2023 · 2 comments
Assignees

Comments

@EricClaeys
Copy link
Collaborator

Requested changes to the flow timer change:

  • Allow the number of images to process to optionally be passed as an argument (to flow_runner.py ?). The default can remain at 10.
  • Add a variable to allsky/variables.sh to define the directory, e.g., ALLSKY_FLOWTIMINGS. This will make it easy for check_allsky.sh to find the directory.
  • How will check_allsky.sh know the names of the files? Should other variables be added, e.g., ALLSKY_FLOWTIMINGS_XXX ?
  • allsky/allsky.sh should rm -fr "${ALLSKY_FLOWTIMINGS}" so we know any files are from the current instance of Allsky.
  • Please make times stored in the files in integer milliseconds.
Alex-developer added a commit to Alex-developer/allsky that referenced this issue May 31, 2023
Alex-developer added a commit that referenced this issue May 31, 2023
@Alex-developer
Copy link
Collaborator

  • Allow the number of images to process to optionally be passed as an argument (to flow_runner.py ?). The default can remain at 10.

Added --flowtimerframes option to flow-runney.py Default is 10

  • Add a variable to allsky/variables.sh to define the directory, e.g., ALLSKY_FLOWTIMINGS. This will make it easy for check_allsky.sh to find the directory.

Added the following variables
ALLSKY_FLOWTIMINGS="${ALLSKY_TMP}/flowtimings"
ALLSKY_FLOWTIMINGS_DAY="${ALLSKY_FLOWTIMINGS}/day-average"
ALLSKY_FLOWTIMINGS_NIGHT="${ALLSKY_FLOWTIMINGS}/night-average"

  • How will check_allsky.sh know the names of the files? Should other variables be added, e.g., ALLSKY_FLOWTIMINGS_XXX ?

See above

  • allsky/allsky.sh should rm -fr "${ALLSKY_FLOWTIMINGS}" so we know any files are from the current instance of Allsky.

New option --cleartimings added to flow-runner.py. This is called from allsky.sh to clear all timing data

  • Please make times stored in the files in integer milliseconds.

Values are now ints

@EricClaeys
Copy link
Collaborator Author

Closing Issue. Changes will be in upcoming point release.
Thanks Alex!!

thomasjacquin added a commit that referenced this issue Jun 15, 2023
* Improved log messages in install.sh

Improve many of the log messages to make troubleshooting easier.

* Update install.sh: shellcheck fix

* Update install.sh: real shellcheck fix

* Fix install.sh bug when updating settings file

Using "mv" to replace a file removes any hard links it has, so use "cp" instead.

* Changes as requested on #2717

* Correct scl/sda on dew heater circuit #2728

* install.sh: fix bug #2 with unlink of settings files

* README.md: mention Ubuntu and fix links

* Delete WebUI.png - not needed

Using an image of the WebUI in documentation folder.

* capture_ZWO.cpp: initial changes for experimental mode

* Remove "#ifdef HISTOGRAM" - it's always used and will be replaced.
* Do more error checking.
* Add "experimental" exposure mode that uses our auto-exposure algorithm at night.  This works well in limited testing but needs more testing.
* Add "useSnapshotMode" for a future addition that uses ZWO's snapshot mode instead of video mode to take a picture.
* Modify many Log() message for better troubleshooting.
* Change indenting in several places to make code easier to read.
* When ping-ponging, add a percent of the number of ping pongs (basically, a unique amount) to the exposure time.  Not sure it helps, but try anyhow.

* Update allsky_common.h: add useExpirementalExposure

* Update allsky_common.h: fix spelling of Experimental

* Update allsky_common.cpp: add experimentalExposure

* Update capture_ZWO.cpp: use global variable

* Update options.json.repo: add experimentalExposure

* Update ASI_functions.cpp: add experimentalExposure

* Update ASI_functions.cpp: fix bug: %s vs %d

* Update allsky_common.cpp: add useExperimentalExposure

* Update capture_ZWO.cpp: fix name misspelling

* Update allsky_common.cpp: display day/night mean settings for ZWO

Also, move some lines to match the order in the WebUI

* Update ASI_functions.cpp: Allow ModeMean and MeanThreshold for ZWO

* Update allsky_common.cpp: align output

* Update allsky_common.cpp: fix: arguments need to be lowercase

* Update capture_ZWO.cpp: no need to check if histogram is NULL

It's now always the array of ints.

* Update capture_ZWO.cpp: move histogram definition to where it's needed

Also no need for "else" after "continue"

* Update capture_RPi.cpp

No need for "else" after "continue"

* Update options.json.repo: default mean targets should be based on camera

* Update allsky.html: remove "RPi only" for Target Mean

Also mention default gains.

* Update functions.sh: bug fix: don't assume SETTINGS_FILE

* Update allsky_common.h: change ZWO mean defaults to decimal

This matches RPi.

* Update allsky_common.cpp: minor wording change

* Update allsky_common.h: set day target mean values to old values

* Update ASI_functions.cpp: see description

* Split "mean" into "daymean" and "nightmean" since their settings are different.
* "Exposure" output to CC file needs to be in ms to match the WebUI.
* Add day/night entries whose defaults are different.

* Update capture_ZWO.cpp: multiply min/max mean by 255

The WebUI uses 0.0 to 1.0 (just like with RPi), but our current ZWO auto-exposure algorithm uses 0 to 255.

* Update createAllskyOptions.php: major updates

* [a-z]_min/_max/_default/_display no longer have [a-z].  This makes it less error prone when adding settings.  These fields' values are obtained from the fields' CC file entries (if present).
* Values for "day_" and "night_" field values get their values from the setting name's entry in the CC file.  For example, "dayexposure" and "nightexposure" have different defaults: "day_default" and "night_default", whereas their min and max values are the same.

* Update options.json.repo: see description

The "[a-z]_min/_max/_default/_display" values no longer have the "[a-z]" which makes it easier and less error prone when adding settings.
When _min/_max/_default/_display are used with a setting whose name begins with "day" or "night", the day and night values are the same in the CC file.

* Update Makefile: bug fix: "sunwait" needs to depend on "patchsunwait"

* Update allsky.html: "Mean Threshold" is also available on ZWO

* Update changeLog.html: add 3 new ZWO settings

* src/Makefile: temporarily break dependency on patchsunwait

It causes the make to fail.

* allsky_common.cpp: remove unnecessary checks

If the target mean is 0.0, the min/max would also result in 0.0, so don't need the checks.

* Update allsky_common.h: update values

* Update allsky_common.cpp: always display target means

* Update allsky_common.cpp: minor wording change

* Update allsky_common.cpp: don't multiple nightExposure by 1000

* Update ASI_functions.cpp: Set default brightness

* Update capture_ZWO.cpp: set defaults, then grab arguments.

* Remove sunwait patch #2738

* Remove sunwait patch #2738

* Update allskySettings.php

* Update options.json.repo: use "integer", "float", and "boolean"

* Update options.json.repo: put space after ":" for consistency

* Update allskySettings.php: handle "boolean" and "percent"

Change "checkbox" which is an html term, with "boolean" which indicates the type of data.

* Update index.php: don't pass WLAN interface name

* Update dashboard_WLAN.php: add function to display a single interface

Determine all the interfaces, then loop through them displaying each one's information.

* Update changeLog.html: add multiple Wi-Fi interfaces supported.

* functions.php fix: don't assume $result[1] exists

Also, add the interface name to status messages in case there are multiple interfaces.

* index.php: Display messages from functions if needed

If initialize_variables() or another global function called by index.php sets an error message, display it.

* Update functions.php: check if numbers used in math are numeric

Without this check, if a user entered a non-numeric value, the math with fail.

* status_messages.php: add $highlight argument

If a caller wants to highlight the table of messages it can set the new $highlight argument which puts a border around the table and add a line at the top and bottom.

* Update index.php: display messages if needed

If an error was generated by a function that's not for a particular page, display the message(s) in "highlight" mode.

* Update allskySettings.php: validate numbers

* Check for legal numbers.
* Reset $key rather than use $originalName to make it more obvious what's happening.
* Display message more efficiently.

* Update functions.php: set daydelay to -1 instead of delay

bug fix

* Update changeLog.html: mention validating numbers

* Update allskySettings.php: state if fractions are allowed or not

* Update custom.css: make alert-danger class more obvious

* Update install.sh: misc changes

* Use new function update_json_file()
* Quote {} as a best practice
* Change mode and group of thumbnails directories.

* Update functions.sh: add update_json_file()

* Automatically fix broken settings links

* Update options.json.repo

* Update allsky_common.cpp

* Update capture_ZWO.cpp

* Update allsky.html: add New Exposure Algorithm

* Update options.json.repo: rename option

* Update allsky_common.cpp: rename option

* Update installUpgradeFunctions.sh: only save 1st line of branch

* Update install.sh: check for multiple branches in .git/config file

* Update variables.sh: add ALLSKY_ABORTEDSAVEIMAGE

* Update functions.sh: add one_instance()

* Update upload.sh: use one_instance() instead of our own code

* Update upload.sh: remove unused variable

* Update timelapse.sh: use one_instance() instead of our own code

* Update functions.sh: use "ps -fp"

So we get the command-line arguments as well.

* Update saveImage.sh: use one_instance()

* check_settings_link(): use settings()

Also, use the full file name in error output.

* install.sh: get convert_settings() working, plus more

* Get convert_settings() working for v2022.03.01.
* Fix broken prior settings file links.

* Rename PRIOR_ALLSKY values to "newStyle" and "oldStyle".
* Get the prior camera type from the prior settings file if it's a "newStyle" file.
* Several indentation changes to make things easier to read.
* Display settings file list in a couple other places for debugging.

* determineCommandToUse(): send errors to stderr

So they appear in the log file.

* SSL clarifications

* saveImage.sh: bug fix: remove aborted image

* functions.sh: run raspistill to check if it works

* Update variables.sh: rename 3 ABORTED... variables

* Update variables.sh: put aborted files in same directory

* Update allsky.sh: use new directory for aborted info

* Update saveImage.sh: new arguments to one_instance() and timelapse.sh

* Use new "--caused-by" argument to one_instance().
* Specify the time to wait for one_instance() so it doesn't wait too long.
* Set AS_TEMPERATURE_C if not already set and there's a temperature.txt file.
* Only call darkCapture.sh if needed.
* Use new arguments to timelapse.sh.

* Update functions.sh: argument changes

* Accept new "--caused-by" argument.
* Don't bother checking that the process in the PID_FILE is the specified one.  --process-name is no longer needed.
* Use new $ALLSKY_ABORTS_DIR.

* Update saveImage.sh: don't use --process-name

* Update upload.sh: argument changes to one_instance()

* Update saveImage.sh: shellcheck fix

* Update darkCapture.sh: changes due to changes in saveImage.sh

* We're only called if taking dark frames, so no need to check.  This changed indentation.
* saveImage.sh updates AS_TEMPERATURE_C as needed so we don't need to.

* Update timelapse.sh: new arguments

* Accept new --lock, --output, and --images argument.
* Updated usage message.
* Updated usage checks.
* Only check for valid DATE and DATE_DIR is needed.

* Update timelapse.sh: new arguments

* Accept new --lock, --output, and --images argument.
* Updated usage message.
* Updated usage checks.
* Only check for valid DATE and DATE_DIR is needed.

* Update timelapse.sh: Fix/improve determination of input and output directories.

Also, make sure the PID_FILE is removed.

* Update timelapse.sh: redo main loop

The prior way of creating the SEQUENCE_DIR files was to use gawk and a couple sub-shells.  The new way which is simpler (and hopefully faster) uses a "while read" loop.

* Update generateForDay.sh: make_thumbnail() and others

* Pass output timelapse file name to timelapse.sh.
* Use global make_thumbnail() instead of our own version.
* Show preferred "--" arguments instead of "-".
* Change some indentation to make code easier to read.

* Update timelapse.sh: shellcheck fix

* Update endOfNight.sh: use preferred "--" arguments

* Update functions.sh: add make_thumbnail()

* Update saveImage.sh: use make_thumbnail()

* Update install.sh: call check_new_exposure_algorithm()

* install.sh: Support restarting where left off

* Add STATUS_* variables to hold installation status in case we need to restart, e.g., after exiting to change locale.
* Used "local" for local variables so it's more obvious what's global.

* Update save_file.php: Add "config" check

* Update install.sh: Add get_connected_cameras()

* Add get_connected_cameras() and call early on to see if there are any connected cameras.
* Add more STATUS_* messages.

* Update install.sh: better way to update STATUS_VARIABLES

* Update install.sh: shellcheck fix

* Update install.sh

* Update install.sh: rename LOCALE to DESIRED_LOCALE

This better reflects its purpose.

* Update install.sh: minor wording changes.

* timelapse.sh: check for missing or 0-length files.

* Update install.sh: use "git rev-parse --abbrev-ref HEAD" to determine branch

It's faster and handles multiple branches.

* Update install.sh: add extra "MORE_STATUS" argument to exit_installation

* Update install.sh: Add more status lines.

* Save status for every Python dependency installed.
* Don't prompt for locale if the prior Allsky had a valid locale.
* Rename "get_locale()" to "get_desired_local()" since that's it's primary function.

* Update install.sh: skip already performed Python dependency installations

* Update install.sh: hopefully last set of changes for restarting installation

* Update functions.php: check for "" lastChanged

* Update allsky.sh: check for unconfigured settings

* Update install.sh: Reset lastChanged to force users to look at configuration

* Update install.sh: convert a few more old settings to new

* Delete flow-runner.py - accidently added to root

The correct one is in "scripts".

* Update functions.sh: improve jq-related commands

* Check for missing "." as first character.
* Add update value to error message.

* Update install.sh: fix the "sed" line to break .json file into field and value

* Makefile: handle change to uname -m for VMs

See comments in Makefile.

* Update changeLog.html: mention bug fix

* Update install.sh: use $CAMERA instead of $CAMERA_TYPE

* functions.sh: Don't do "ls -l" of FILE

Often the jq fails because FILE doesn't exist, and jq outputs that error so there's no need to do "ls" of the file since it will also fail.

* Update functions.sh: better error messages and return codes

If we can't find or read the file, return with EXIT_ERROR_STOP since that's a fatal problem.

* Update install.sh: 2 bug fixes

* Fix misspelled "STATUS_FILE_TEMP"
* Use PRIOR_SETTINGS_FILE instead of SETTINGS_FILE

* Update install.sh: Save status on interrupt

* Also fix bug saving STATUS_CODE (was called STATUS).

* Update variables.sh: add ALLSKY_UPTIME_SINCE

* Update functions.sh: add check_for_reboot

Returns 0 is use doesn't need to reboot, otherwise 1.

* Update functions.sh: reverse return code

* Update allsky.sh: check if a reboot is needed

* Update installUpgradeFunctions.sh: add get_allsky_branch

It uses a git command to get the branch instead of a file we maintain.

* Update installUpgradeFunctions.sh: shellcheck fix

* Update functions.sh: list invalid field

* Update check_allsky.sh: check for reboot and config

* Also check for errors from settings()

* Update install.sh: indicate when reboot needed.

* Update install.sh: shellcheck fix

* Update functions.php: set lastChanged to "" if not there or blank

* Update allskySettings.php: check for $lastChanged === ""

* Update allsky.sh: fix doExit arguments

* Update functions.sh: output reason for stop

* Update functions.sh: fix variable name

* Update allsky.sh: add system messages

* settings() bug fix

json arrays, e.g., "keys[]" can NOT start with a period, but other fields must.

* removeBadImages.sh: shorten message so it fits in image

The "X consecutive..." message doesn't quite fit in the screen so remove a newline.

* Update variables.sh

* Update install.sh

* Update functions.sh

* Update upgrade.sh

* Update variables.sh: delete ALLSKY_BRANCH_FILE

* Update installUpgradeFunctions.sh: see description

* Remove get_branch() since it used ALLSKY_BRANCH_FILE which is no longer used.
* Rename get_allsky_branch() to get_branch().

* Update install.sh: don't append "/" to PRIOR_WEBSITE

It's no longer needed.

* Update install.sh: use get_branch() instead of get_allsky_branch()

And don't append "/" to directory name

* Update upgrade.sh: use get_branch(), not get_allsky_branch()

* Update check_allsky.sh: use get_branch(), not get_allsky_branch()

* Update check_allsky.sh: shellcheck fix

* Update install.sh

* Update install.sh: improve look of "Restoring prior:" output

* Update allsky_common.h: add variables for next version

* Update allsky_common.h: change night values

* Update mode_mean.h: change two comments

* Update allsky_common.cpp: improve Log() messages

* Update allsky_common.cpp: replace spaces with tab

* Update options.json.repo: "Brightness" is deprecated

* Update ASI_functions.cpp: improve Log() messages

Plus add dayThreshold and nightThreshold for future use

* Update capture_RPi.cpp: better Log() messages

* Update capture_ZWO.cpp: better Log() messages

Also:
* Rename "acceptable" to "acceptableMean" to better reflect what it is.
* Improve indentation to make code easier to read.

* Update allsky_common.h

* Rename and move "meanValue"

* Update capture_RPi.cpp

* Update capture_ZWO.cpp

* Update mode_mean.h: remove meanValue

* Update mode_mean.h: remove mean_threshold

* Update capture_RPi.cpp: remove myModeMeanSetting.mean_threshold

* Update mode_mean.cpp: rename mean_threshold to currentMean_threshold

* Update mode_mean.h: remove mean_p[012]

* Update mode_mean.cpp: use cg->myModeMeanSetting instead of currentModeMeanSetting

* Update capture_RPi.cpp: deleted unneeded lines

* Update allsky.html: Brightness has been deprecated

* install.sh: add a couple log entries

Also, increase the size of a whiptail box so the question is visible.

* Update install.sh: use convert_json_to_tabs

* Update functions.sh: add convert_json_to_tabs()

* Update allsky.sh: use convert_json_to_tabs

This also allows us to replace the "for KEY in" with a grep/sed.

* check_allsky.sh: remove duplicate checks

---------

Co-authored-by: Eric Claeys <[email protected]>
Co-authored-by: Alex Greenland <[email protected]>
Co-authored-by: Alex <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants