[WIP][8.0]refactor gfal2 and storageElement #6127
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DIRACGridBot
added
the
alsoTargeting:integration
chaen
force-pushed
the
rel-v7r3_FEAT_refactorGfal2
branch
from
3854d7e
to
898a174
Compare
chaen
force-pushed
the
rel-v7r3_FEAT_refactorGfal2
branch
3 times, most recently
from
93a523d
to
38496cd
Compare
andresailer
reviewed
Jun 3, 2022
protocolLists for TURL and 3RD_PARTY Turl, file is not default in SRM anymore
…d protocolSectionName
…s take 'protocolSection' as parameter instead of 'plugin'
chaen
force-pushed
the
rel-v7r3_FEAT_refactorGfal2
branch
from
c4d7699
to
a7ed196
Compare
fstagni
reviewed
Jun 21, 2022
| @@ -458,6 +458,7 @@ def __loadCFGFiles(self): | |||
| if "DIRACSYSCONFIG" in os.environ: | |||
| diracSysConfigFiles = os.environ["DIRACSYSCONFIG"].replace(" ", "").split(",") | |||
| for diracSysConfigFile in reversed(diracSysConfigFiles): | |||
| gLogger.debug("Loading file from DIRACSYSCONFIG %s" % diracSysConfigFile) | |||
There was a problem hiding this comment.
Suggested change
| gLogger.debug("Loading file from DIRACSYSCONFIG %s" % diracSysConfigFile) | |
| gLogger.debug(f"Loading file from DIRACSYSCONFIG {diracSysConfigFile}") |
| @@ -116,8 +116,6 @@ def __init__(self, storageName, parameters): | |||
| self.stageTimeout = gConfig.getValue("/Resources/StorageElements/StageTimeout", 12 * 60 * 60) | |||
| # gfal2Timeout, amount of time it takes until an operation times out | |||
| self.gfal2Timeout = gConfig.getValue("/Resources/StorageElements/GFAL_Timeout", 100) | |||
| # set the gfal2 default protocols, e.g. used when trying to retrieve transport url | |||
| self.defaultLocalProtocols = gConfig.getValue("/Resources/StorageElements/DefaultProtocols", []) | |||
There was a problem hiding this comment.
This is referenced also in the documentation, it should be removed from there too.
| @@ -51,16 +56,10 @@ def __init__(self, storageName, parameters): | |||
|
|
|||
| self.gfal2requestLifetime = gConfig.getValue("/Resources/StorageElements/RequestLifeTime", 100) | |||
|
|
|||
| self.__setSRMOptionsToDefault() | |||
| self.protocolsList = self.protocolParameters["OutputProtocols"] | |||
| self.log.debug("GFAL2_SRM2Storage: protocolsList = %s" % self.protocolsList) | |||
There was a problem hiding this comment.
Suggested change
| self.log.debug("GFAL2_SRM2Storage: protocolsList = %s" % self.protocolsList) | |
| self.log.debug(f"GFAL2_SRM2Storage: protocolsList = {self.protocolsList}") |
chaen
force-pushed
the
rel-v7r3_FEAT_refactorGfal2
branch
from
861242f
to
c411829
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Although this PR has the potential to break things, I do not think it will. If you were in a situation that would break, it means:
It contains 4 main changes:
Test_StorageFactorypluginName(which is the effectively used object to perform operations) and theprotocolSectionName(which is the name of the section in the CS used to configure a given protocol).DefaultProtocolsoption in theStorageElementssection of the CS is unused anymore byGFAL2_SRM2as redundant withoutputProtocolsHistorically, we never really had to deal with such cases, but now we have situations where the same storage plugin can be used for different protocols, and hence corresponds to two different protocol sections. We thus need to be able to make the difference.
It contains an API change for the
StorageElementobject (that will go in the wiki):pluginparameter is replaced withprotocolSection@andresailer I guess you are the most appropriate to review that. This change would be best suited for v8.0, but given that it risk of not coming out anytime soon, I am almost tempting to tag it
v7r4...In a second stage, I will tackle #4744 once for all
BEGINRELEASENOTES
*Resources
CHANGE: GFA2_SRM2 does not use anymore StorageElements/DefaultProtocols option in the CS (relies on specific OutputProtocols config)
CHANGE: explicit distinction between PluginName and ProtocolSection name in all the StorageElement components
CHANGE: API change for the StorageElement. See #6127 for details
ENDRELEASENOTES