How to check if a file on a datastore exists? #1061
-
|
As title says. Let's say there's a datastore Basically, we stumble upon a vsphere bug where it silently disconnects a CD-rom if ISO file it's using doesn't exist. So there's no error or anything, it just silently changes the state. As a workaround we want to check manually if the file exists before powering on the machine. I've searched over a bunch of different sources, documentation, tried looking Worth adding that the datastore is read-only in my case, so some kind of hack as attempting to rename file to the name it already has or change it's permissions to the ones already present will likely not work. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 4 replies
-
|
You can use the searchSpec = vim.host.DatastoreBrowser.SearchSpec()
searchSpec.matchPattern = ["my.iso"]
task = browser.Search("[vsanDatastore] 39cce864-d1d5-b67b-a2b8-88aedd01f1d5", searchSpec)I hacked up a complete example below. It is interesting that both VSAN paths with UUIDs work and also the friendly names say of a virtual machine. To run this create .env file in the same folder with VSPHERE_SERVER, VSPHERE_USER and VSPHERE_PASSWORD values (or set those in the environment): import logging
import os
import sys
import time
from typing import List, Dict
from dotenv import load_dotenv
from pyVim.connect import SmartConnect, Disconnect
from pyVmomi import vim, vmodl
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s - %(levelname)s - %(message)s',
handlers=[logging.StreamHandler(stream=sys.stdout)])
def get_property(name: str, props: List[vmodl.DynamicProperty]) -> object:
"""gets property by name from a property set"""
for prop in props:
if prop.name == name:
return prop.val
def get_datastores(si: vim.ServiceInstance) -> Dict[vim.Datastore, str]:
"""Get all datastores and their names using container view and property
collector"""
# pylint: disable=broad-exception-caught
content = si.content
viewManager = content.viewManager
pc = content.propertyCollector
token = None
view = viewManager.CreateContainerView(content.rootFolder,
[vim.Datastore],
True)
try:
opts = vmodl.query.PropertyCollector.RetrieveOptions()
opts.maxObjects = 1000
filter_spec = vmodl.query.PropertyCollector.FilterSpec()
property_spec = vmodl.query.PropertyCollector.PropertySpec()
property_spec.type = vim.Datastore
property_spec.pathSet = ['name']
filter_spec.propSet = [property_spec]
obj_spec = vmodl.query.PropertyCollector.ObjectSpec()
obj_spec.obj = view
ts = vmodl.query.PropertyCollector.TraversalSpec()
ts.name = "traverseEntities"
ts.path = "view"
ts.type = vim.view.ContainerView
ts.skip = False
obj_spec.selectSet = [ts]
filter_spec.objectSet = [obj_spec]
filter_spec.reportMissingObjectsInResults = False
res = pc.RetrievePropertiesEx([filter_spec], opts)
token = res.token
objects = res.objects
while token:
res = pc.ContinueRetrievePropertiesEx(token)
token = res.token
objects = objects + res.objects
datastores = {}
for obj in objects:
name = get_property("name", obj.propSet)
datastores[obj.obj] = name
return datastores
finally:
if token:
try:
pc.CancelRetrievePropertiesEx(token)
except Exception as ex:
logging.error("Cannot cancel retrieve properties loop. \
Dismiss the session to recover. %s", ex)
view.Destroy()
def wait_task(task: vim.Task) -> object:
""" Waits for task to complete """
info = task.info
while info.state in ["queued", "running"]:
print("Waiting task....")
time.sleep(1)
info = task.info
if info.state == "error":
raise info.error
return info.result
def search_ds_path(browser: vim.host.DatastoreBrowser, base: str, item: str) \
-> vim.host.DatastoreBrowser.FileInfo:
""" Invoke vim.host.DatastoreBrowser.search and wait for task to
complete. """
searchSpec = vim.host.DatastoreBrowser.SearchSpec()
searchSpec.matchPattern = [item]
task = browser.Search(base, searchSpec)
res: vim.host.DatastoreBrowser.SearchResults = wait_task(task)
logging.debug("Search result %s", res)
if not res.file:
return None
return res.file[0]
def find_ds_by_name(si, ds_name):
""" Find datastore by name """
datastores = get_datastores(si)
for ds, name in datastores.items():
if name == ds_name:
return ds
return None
def file_exists(si: vim.ServiceInstance, path: str) -> bool:
"""Check if file path exists."""
if not path or len(path) <= 4 or path[0] != "[":
logging.debug("Invalid path: %s", path)
return False
idx = path.find("] ")
if idx < 0:
logging.debug("Cannot datastore end name indication '] ' in %s", path)
return False
ds_name = path[1:idx]
path_segments = (path[idx+2:]).split("/")
datastore = find_ds_by_name(si, ds_name)
if not datastore:
logging.debug("datastore name not found %s", ds_name)
return False
browser = datastore.browser
base_path = f"[{ds_name}]"
if len(path_segments) > 1:
base_path += " " + '/'.join(path_segments[:-1])
item = path_segments[-1]
try:
res = search_ds_path(browser, base_path, item)
except vim.fault.FileNotFound:
logging.debug("Base path is not found: %s", base_path)
return False
if not res:
logging.debug("No matches found for '%s' in '%s'",
item,
base_path)
logging.debug("Found: %s", res)
return True
def main():
"""tries to locate several files by name"""
files = ["[nfs] esx1_scratch/log/attestd.log",
"[vsanDatastore] 39cce864-d1d5-b67b-a2b8-88aedd01f1d5/hugo.vmx",
"[vsanDatastore] hugo/hugo.vmx",
"[vsanDatastore] hugo",
]
load_dotenv()
si = SmartConnect(host=os.environ["VSPHERE_SERVER"],
user=os.environ["VSPHERE_USER"],
pwd=os.environ["VSPHERE_PASSWORD"],
disableSslCertValidation=True)
try:
res = {}
for filename in files:
res[filename] = file_exists(si, filename)
for filename, exists in res.items():
logging.info("%s - %s", exists, filename)
finally:
Disconnect(si)
if __name__ == "__main__":
main() |
Beta Was this translation helpful? Give feedback.
-
But doesn't that search whole datastore for a single file? This is not very good, because depending on the amount of files and the speed of the datastore that may take long time. In my case the datastore is an NFS, so it's gonna be slower than even an HDD disk. Sorry, but no, I would not use that. |
Beta Was this translation helpful? Give feedback.
-
|
Double thank you for the complete code! That turned out to be more complicated than I expected, so I would have spent on this too much time, wouldn't it be for your code… |
Beta Was this translation helpful? Give feedback.
-
|
For future readers, I replaced the def wait_task_w_return(si: vim.ServiceInstance, task: vim.Task):
""" Similar to WaitForTask, but returns the execution result """
pc = si.content.propertyCollector
progressUpdater = ProgressUpdater(task, None)
progressUpdater.Update('created')
filter = CreateFilter(pc, task)
version = None
# Loop looking for updates till the state moves to a completed state.
while task.info.state not in ('success', 'error'):
try:
version, _ = GetTaskStatus(task, version, pc)
progressUpdater.UpdateIfNeeded()
except vmodl.fault.ManagedObjectNotFound as e:
print("Task object has been deleted: %s" % e.obj)
break
DestroyFilter(filter)
info = task.info
state = info.state
if state == "error":
progressUpdater.Update(f'error: {info.error}')
raise info.error
elif state == "success":
progressUpdater.Update('completed')
else:
raise Exception(f'Task was not finished. Task info: {info}')
return info.result |
Beta Was this translation helpful? Give feedback.
-
|
@karaatanassov I think you accidentally closed the discussion. I can't seem to re-open it. |
Beta Was this translation helpful? Give feedback.
-
|
No. This API searches the folder path. In the example we are aksing - does the folder "[vsanDatastore] 39cce864-d1d5-b67b-a2b8-88aedd01f1d5" contain an entry "my.iso". You can go with more complex paths. For example my content library stores .iso images in VSAN datastore. The path is so then to call the API I need to split it in file name and folder path like so On my test set up the above takes less then a second to complete. So I am pretty sure it does not traverse the whole datastore. There is searchSubFolders method that I suppose will be taking more time if issued at the datastore root path e.g. "[vsanDatastore]" With vCenter 8.0.2 there is new API to check the lock status of a single path. So then you do not have to split the path into base path and file name. You can try that but there is complication that is requires ESX host in addition to path: vim.FileManager::queryFileLockInfo Sorry about closing the discussion I thought it is an issue for some reason. |
Beta Was this translation helpful? Give feedback.
-
|
Aah, I see, thank you very much, then I think that will work for me! ✔ |
Beta Was this translation helpful? Give feedback.
No. This API searches the folder path. In the example we are aksing - does the folder "[vsanDatastore] 39cce864-d1d5-b67b-a2b8-88aedd01f1d5" contain an entry "my.iso".
You can go with more complex paths. For example my content library stores .iso images in VSAN datastore. The path is
so then to call the API I need to split it in file name and folder path like so