v3.00.54 - Wololo!, Voluble Status and Partitioned Flows

What's new

• GeoJSON style filtering support
• Implicit FTPS support
• PAHO MQTT V2 support
• Default queueName change
• Partitioned transfers now working and included in CI/CD testing (sr3 is now a super-set of v2, no remaining functionality regressions)
• The --wololo option speeds up the v2 to sr3 converter substantially.
• In place of just running or stopped in sr3 status, there is now: idle, lagging, slow, retry, rejecting, cpuSlow indicating various ways flows could be in difficulty or require tuning.
• HTTP scheduled flows can now query the resource using metadata calls for more effective polling.
• Many fixes for issues people ran into during deployments.
• Also fixes for problems with statehost setting (important for HPC deployments.)

What's Changed

• Updates to NASA Auth plugins and examples by @reidsunderland in #1042
• Issue1036 - Add unit tests for AM code by @andreleblanc11 in #1044
• Fix sanity not starting missing instances by @reidsunderland in #1053
• Build Unit test boilerplate by @gcglinton in #1052
• multi-instances will be partial, not missing by @petersilva in #1059
• Dev doc refactor release instructions by @petersilva in #1057
• to convert overwriting, one must Wololo by @petersilva in #1055
• Fix #1008 and update documentation by @mshak2 in #1056
• Fix bug caused by different variable names by @reidsunderland in #1060
• Unit Test Engineering wmo00 by @petersilva in #1048
• Unit Test Engineering of the sarracenia/config.py module. by @petersilva in #1058
• Fixes search function and logo, and adds favicon for doc website by @mshak2 in #1062
• Fix #1064 by adding try/except by @reidsunderland in #1065
• #980 Expand scope of isProblem. by @andreleblanc11 in #1046
• #1036 Add more AM unit tests for latest bug findings by @andreleblanc11 in #1068
• Issue824 morestates and issue966 and... other stuff by @petersilva in #1067
• please stop infinite loop by calling self by @petersilva in #1070
• disabled configs were showing as "unknown" by @petersilva in #1073
• Fix #1074 by @andreleblanc11 in #1075
• Bug fix for AM sender by @andreleblanc11 in #1078
• Doc updates by @petersilva in #1072
• Fix mkdir failures on hpc by @petersilva in #1076
• Fix #1079 by @andreleblanc11 in #1081
• remove plugins that are useless and will not run. by @petersilva in #1082
• Add post_baseUrl to delete on post, changes to logging by @reidsunderland in #1050
• adding new_pathWrite() as per Reid suggestion by @petersilva in #1049
• Issue1080 2 by @petersilva in #1083
• Add scheduled flow with http metadata by @reidsunderland in #1084
• Issue1035 retry too much, reduce cpu to some extent... by @petersilva in #1086
• fix typo rel_relPath -> new_relPath by @reidsunderland in #1092
• Scheduled flow with HTTP metadata: add variable expansion for the path by @reidsunderland in #1088
• honour statehost in one mkdir case by @petersilva in #1087
• fix #1091 download and/or sender plugin error crashes by @petersilva in #1093
• Doc feedback by @petersilva in #1094
• issue #982 detail in cleanup was not working when statehost set by @petersilva in #1096
• more robust validation of return value from destfnscript by @petersilva in #1085
• Add directory to send, etc. errors by @reidsunderland in #1099
• only run after_post after we actually post by @petersilva in #1101
• fix #1095 crash in sender when size missing from message by @petersilva in #1102
• Issue1089 fileAgeMin and fileAgeMax should be honoured by all components by @petersilva in #1100
• Finalize Azure Transfer tests by @gcglinton in #1051
• fix crash where 'dictionary changed size during iteration' by @reidsunderland in #1109
• Change chmod -> utime, i think it was a typo? by @reidsunderland in #1108
• Issue1104 msg rate cpu by @petersilva in #1105
• Random fixes by @petersilva in #1103
• Restoring blockmanifest transfers by @petersilva in #1110
• Issue1097 fractional rates by @petersilva in #1106
• #1115 stopped flows were marked cpuS. fix by @petersilva in #1116
• Adjust column widths, shorten units. by @petersilva in #1117
• Enable blocked tests by @petersilva in #1112
• fix #1002 putAccellerated when file names have colons by @petersilva in #1118
• Implement support for implicit FTPS on port 990 by @reidsunderland in #1122
• Fix the NASA CMR plugin, bug when using the other data source by @reidsunderland in #1123
• fix test for NASA CMR rename by @reidsunderland in #1124
• Fix logging for default poll by @reidsunderland in #1125
• Paho mqtt v2 support by @petersilva in #1119
• Issue33 - default queue name change. by @petersilva in #1128
• Issue1043 geojson by @Jeanc16888 in #1130
• Release doc touchups by @petersilva in #1136
• fix #1089 - created getIDStr function for Sarracenia.Message by @petersilva in #1135
• Fix #1133 by @andreleblanc11 in #1134
• Adding copyright explanation by @petersilva in #1139
• Authentication plugin refactor by @reidsunderland in #1142
• Fix for #1141 by @mshak2 in #1146

New Contributors

• @Jeanc16888 made their first contribution in #1130

Full Changelog: v3.0.53...v3.00.54

Packages Available:

Ubuntu PPA: https://launchpad.net/~ssc-hpc-chp-spc/+archive/ubuntu/metpx

PyPI: https://pypi.org/project/metpx-sr3/3.0.54/

• RPM package for Redhat Version 8 (rh8) and 9 (rh9) attached here, install with rpm -ivh ... note:

  • Some dependencies are not readily available. May have to obtain packages
    from pip to get a fully functional installation. Verify with sr3 features after installation, to determine if everything needed is available.

• self-extracting windows installer also attached.