如何从Azure Blob数据存储中获取Python pathlib路径？ [英] How get a Python pathlib Path from an Azure blob datastore?

问题描述

我正在尝试对AzureML中的 torch.utils.data.DataLoader 进行一些自定义操作，但是无法直接从我的实例化它azureml.core.Datastore ：

I am trying to do some custom manipulation of a torch.utils.data.DataLoader in AzureML but cannot get it to instantiate directly from my azureml.core.Datastore :

ws = Workspace( # ... etc ... )
ds = Datastore.get(ws, datastore_name='my_ds')
am = ds.as_mount()

# HOW DO I GET base_path, data_file from am?

dataloader = DataLoader(
        ListDataset(base_path, data_file), #... etc...
    )

am.path（）的值为 $ AZUREML_DATAREFERENCE_my_ds，但我无法弄清楚如何从构造函数转到 ListDataset 的 pathlib.Path 。我尝试过的东西包括 Path（am.path（））和 Path（os.environ [am.path（）]）但它们似乎不起作用。

The value of am.path() is "$AZUREML_DATAREFERENCE_my_ds" but I cannot figure out how to go from that to a pathlib.Path as is expected by the constructor to ListDataset. Things I've tried include Path(am.path()) and Path(os.environ[am.path()]) but they don't seem to work.

很明显，有一些答案，因为：

It's clear that there's some answer, since :

script_params = {
    '--base_path': ds.as_mount(), 
    '--epochs': 30,  
    '--batch_size' : 16,  
    '--use_cuda': 'true'
}
torch = PyTorch(source_directory='./',
                        script_params=script_params,
                        compute_target=compute_target,
                        entry_script='train.py',
                        pip_packages=packages,
                        use_gpu=True) 

似乎可以创建合法对象。

seems to create a legit object.

推荐答案

您也许可以尝试使用 DataPath类。它公开了诸如 path_on_datastore 之类的属性，这些属性可能就是您要查找的路径。

You can perhaps try using the DataPath class. It exposes attributes such as path_on_datastore which might be the path you're looking for.

从您的DataReference对象，即变量 am ;您可以使用 create_from_data_reference（）方法。

To construct this class from your DataReference object i.e. variable am; you can use create_from_data_reference() method.

示例：

ds = Datastore.get(ws, datastore_name='my_ds')
am = ds.as_mount()

dp = DataPath().create_from_data_reference(am)

base_path = dp.path_on_datastore

这篇关于如何从Azure Blob数据存储中获取Python pathlib路径？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！