Using AzCopy to Migrate Local Files to Azure blob Storage with SMB Permission - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Tuesday, August 9, 2022

Using AzCopy to Migrate Local Files to Azure blob Storage with SMB Permission

AzCopy is a command-line utility that you can use to copy blobs or files to or from a storage account. This article helps you download AzCopy, connect to your storage account, and then transfer data.

Related Post:


Download

First, download the AzCopy V10 executable file to any directory on your computer. AzCopy V10 is just an executable file, so there's nothing to install.


Authentication

1. Azure AD (OAuth Authentication)
azcopy login --tenant-id=<tenant-id>

2. SAS (Recommended)
- PowerShell to run

Problem:
  • Token generated by Azure AD might have some problem in Windows. 
  • If use CMD, there is a problem for SAS % symbol. 

Run AzCopy

For convenience, consider adding the directory location of the AzCopy executable to your system path for ease of use. That way you can type azcopy from any directory on your system.

If you choose not to add the AzCopy directory to your path, you'll have to change directories to the location of your AzCopy executable and type azcopy or .\azcopy in Windows PowerShell command prompts.


Usage: azcopy copy [source] [destination]



You can provide authorization credentials by using Azure Active Directory (AD), or by using a Shared Access Signature (SAS) token.

Use this table as a guide:

Storage typeCurrently supported method of authorization
Blob storageAzure AD & SAS
Blob storage (hierarchical namespace)Azure AD & SAS
File storageSAS only

Option 1: Use Azure Active Directory

This option is available for blob Storage only. By using Azure Active Directory, you can provide credentials once instead of having to append a SAS token to each command.

Option 2: Use a SAS token

You can append a SAS token to each source or destination URL that use in your AzCopy commands.

This example command recursively copies data from a local directory to a blob container. A fictitious SAS token is appended to the end of the container URL.

AzCopy
azcopy copy "C:\local\path" "https://account.blob.core.windows.net/mycontainer1/?sv=2018-03-28&ss=bjqt&srt=sco&sp=rwddgcup&se=2019-05-01T05:01:17Z&st=2019-04-30T21:01:17Z&spr=https&sig=MGCXiyEzbtttkr3ewJIh2AR8KrghSy1DGM9ovN734bQF4%3D" --recursive=true

To learn more about SAS tokens and how to obtain one, see Using shared access signatures (SAS).


To see a list of commands, type azcopy -h and then press the ENTER key.

To learn about a specific command, just include the name of the command (For example: azcopy list -h).

Inline help


Usage: azcopy sync



Use in Azure Storage Explorer

Storage Explorer uses AzCopy to perform all of its data transfer operations. You can use Storage Explorer if you want to leverage the performance advantages of AzCopy, but you prefer to use a graphical user interface rather than the command line to interact with your files.

Storage Explorer uses your account key to perform operations, so after you sign into Storage Explorer, you won't need to provide additional authorization credentials.




Create a user delegation SAS with PowerShell


About the user delegation SAS

A SAS token for access to a container or blob may be secured by using either Azure AD credentials or an account key. A SAS secured with Azure AD credentials is called a user delegation SAS, because the OAuth 2.0 token used to sign the SAS is requested on behalf of the user.

Microsoft recommends that you use Azure AD credentials when possible as a security best practice, rather than using the account key, which can be more easily compromised. When your application design requires shared access signatures, use Azure AD credentials to create a user delegation SAS for superior security. For more information about the user delegation SAS, see Create a user delegation SAS.


Install the PowerShell module

1 Uninstall any previous installations of Azure PowerShell:

    • Remove any previous installations of Azure PowerShell from Windows using the Apps & features setting under Settings.
    • Remove all Azure modules from %Program Files%\WindowsPowerShell\Modules.

    2 Make sure that you have the latest version of PowerShellGet installed. Open a Windows PowerShell window, and run the following command to install the latest version:

    You might get following error from some old OS which might not install .net framework version :

    PS C:\Tools> Install-Module PowerShellGet -Repository PSGallery -Force
    Get-PSGalleryApiAvailability : PowerShell Gallery is currently unavailable.  Please try again later.
    At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1711 char:9
    +         Get-PSGalleryApiAvailability -Repository $Repository
    +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [Get-PSGalleryApiAvailability], InvalidOperationException
        + FullyQualifiedErrorId : PowerShellGalleryUnavailable,Get-PSGalleryApiAvailability
    
    
    NuGet provider is required to continue
    PowerShellGet requires NuGet provider version '2.8.5.201' or newer to interact with NuGet-based repositories. The NuGet
     provider must be available in 'C:\Program Files\PackageManagement\ProviderAssemblies' or
    'C:\Users\jon\AppData\Local\PackageManagement\ProviderAssemblies'. You can also install the NuGet provider by running
    'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force'. Do you want PowerShellGet to install and import
     the NuGet provider now?
    [Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): Y
    WARNING: Unable to download from URI 'https://go.microsoft.com/fwlink/?LinkID=627338&clcid=0x409' to ''.
    WARNING: Unable to download the list of available providers. Check your internet connection.
    PackageManagement\Install-PackageProvider : No match was found for the specified search criteria for the provider
    'NuGet'. The package provider requires 'PackageManagement' and 'Provider' tags. Please check if the specified package
    has the tags.
    At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:7405 char:21
    + ...     $null = PackageManagement\Install-PackageProvider -Name $script:N ...
    +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidArgument: (Microsoft.Power...PackageProvider:InstallPackageProvider) [Install-Pac
       kageProvider], Exception
        + FullyQualifiedErrorId : NoMatchFoundForProvider,Microsoft.PowerShell.PackageManagement.Cmdlets.InstallPackagePro
       vider
    
    PackageManagement\Import-PackageProvider : No match was found for the specified search criteria and provider name
    'NuGet'. Try 'Get-PackageProvider -ListAvailable' to see if the provider exists on the system.
    At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:7411 char:21
    + ...     $null = PackageManagement\Import-PackageProvider -Name $script:Nu ...
    +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidData: (NuGet:String) [Import-PackageProvider], Exception
        + FullyQualifiedErrorId : NoMatchFoundForCriteria,Microsoft.PowerShell.PackageManagement.Cmdlets.ImportPackageProv
       ider
    
    WARNING: Unable to download from URI 'https://go.microsoft.com/fwlink/?LinkID=627338&clcid=0x409' to ''.
    WARNING: Unable to download the list of available providers. Check your internet connection.
    PackageManagement\Get-PackageProvider : Unable to find package provider 'NuGet'. It may not be imported yet. Try
    'Get-PackageProvider -ListAvailable'.
    At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:7415 char:30
    + ... tProvider = PackageManagement\Get-PackageProvider -Name $script:NuGet ...
    +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : ObjectNotFound: (Microsoft.Power...PackageProvider:GetPackageProvider) [Get-PackageProvi
       der], Exception
        + FullyQualifiedErrorId : UnknownProviderFromActivatedList,Microsoft.PowerShell.PackageManagement.Cmdlets.GetPacka
       geProvider
    
    Install-Module : NuGet provider is required to interact with NuGet-based repositories. Please ensure that '2.8.5.201'
    or newer version of NuGet provider is installed.
    At line:1 char:1
    + Install-Module PowerShellGet -Repository PSGallery -Force
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [Install-Module], InvalidOperationException
        + FullyQualifiedErrorId : CouldNotInstallNuGetProvider,Install-Module
    
    PS C:\Tools>
    

    Install .NET Framework 4.8 from https://dotnet.microsoft.com/en-us/download/dotnet-framework/net48


    3 Run this command

    Install-Module PowerShellGet -Repository PSGallery -Force

    4 Close and reopen the PowerShell window after installing PowerShellGet.  Install the latest version of Azure PowerShell:

    Install-Module Az -Repository PSGallery -AllowClobber


    5 Run the following command to install the latest version of the Azure Storage PowerShell module:

    Install-Module -Name Az.Storage -Repository PSGallery -Force


    6 Close and reopen the PowerShell window. To check which version of the Az.Storage module is installed, run the following command:

    Get-Module -ListAvailable -Name Az.Storage -Refresh


    Sign in to Azure PowerShell with Azure AD




    PS C:\WINDOWS\system32> Set-ExecutionPolicy Bypass CurrentUser
    
    Execution Policy Change
    The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
    you to the security risks described in the about_Execution_Policies help topic at
    https:/go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
    [Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
    PS C:\WINDOWS\system32> Import-Module Az.Accounts
    

    Connect-AzAccount





    When you create a user delegation SAS with Azure PowerShell, the user delegation key that is used to sign the SAS is created for you implicitly. The start time and expiry time that you specify for the SAS are also used as the start time and expiry time for the user delegation key.

    Because the maximum interval over which the user delegation key is valid is 7 days from the start date, you should specify an expiry time for the SAS that is within 7 days of the start time. The SAS is invalid after the user delegation key expires, so a SAS with an expiry time of greater than 7 days will still only be valid for 7 days.

    To create a user delegation SAS for a container or blob with Azure PowerShell, first create a new Azure Storage context object, specifying the -UseConnectedAccount parameter. The -UseConnectedAccount parameter specifies that the command creates the context object under the Azure AD account with which you signed in.

    Remember to replace placeholder values in angle brackets with your own values:



    $ctx = New-AzStorageContext -StorageAccountName <storage-account> -UseConnectedAccount


    Example:
    $ctx = New-AzStorageContext -StorageAccountName fileshareca -UseConnectedAccount



    Create a user delegation SAS for a blob

    To return a user delegation SAS token for a blob, call the New-AzStorageBlobSASToken command, passing in the Azure Storage context object that you created previously.

    The following syntax returns a user delegation SAS for a blob. The example specifies the -FullUri parameter, which returns the blob URI with the SAS token appended. Remember to replace the placeholder values in brackets with your own values:


    New-AzStorageBlobSASToken -Context $ctx `
        -Container <container> `
        -Blob <blob> `
        -Permission racwd `
        -ExpiryTime <date-time>
        -FullUri
    

    The user delegation SAS URI returned will be similar to:


    https://storagesamples.blob.core.windows.net/sample-container/blob1.txt?sv=2018-11-09&sr=b&sig=<sig>&skoid=<skoid>&sktid=<sktid>&skt=2019-08-06T21%3A16%3A54Z&ske=2019-08-07T07%3A00%3A00Z&sks=b&skv=2018-11-09&se=2019-08-07T07%3A00%3A00Z&sp=racwd
    









    PS C:\temp> ./azcopy.exe copy "J:\1992\" "https://fileshareca.file.core.windows.net/fs2/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2022-08-09T09%3A56%3A06Z&st=2022-08-09T01%3A56%3A06Z&spr=https&sig=C9kasdjfiekdkiIFKJFi0389KFKLKJjidfudf3D" --overwrite=prompt --from-to=LocalFile --follow-symlinks --put-md5 --follow-symlinks --preserve-smb-info=true --disable-auto-decoding=false --recursive --log-level=INFO
    INFO: Please note: the preserve-permissions flag is set to false, thus AzCopy will not copy SMB ACLs between the source and destination. To learn more: https://aka.ms/AzCopyandAzureFiles.
    INFO: Scanning...
    INFO: Any empty folders will be processed, because source and destination both support folders. For the same reason, properties defined on folders will be processed
    
    Job fb8eff17-2283-3248-63de-91dd1a3c5100 has started
    Log file is located at: C:\Users\jon\.azcopy\fb8eff17-2283-3248-63de-91dd1a3c5100.log
    
    INFO: Trying 4 concurrent connections (initial starting point)
    INFO: Trying 16 concurrent connections (seeking optimum)
    INFO: Trying 4 concurrent connections (backing off)
    INFO: Trying 8 concurrent connections (seeking optimum)
    76.4 %, 341 Done, 0 Failed, 2 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 12.0535 (Disk may be limiting spee80.3 %, 341 Done, 0 Failed, 2 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 13.1873 (Disk may be limiting speeINFO: Trying 4 concurrent connections (backing off)                                                                     
    80.3 %, 341 Done, 0 Failed, 2 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 13.1873 (Disk may be limiting spee94.4 %, 342 Done, 0 Failed, 1 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 6.6049                            100.0 %, 343 Done, 0 Failed, 0 Pending, 0 Skipped, 343 Total,
    
    
    Job fb8eff17-2283-3248-63de-91dd1a3c5100 summary
    Elapsed Time (Minutes): 0.6026
    Number of File Transfers: 180
    Number of Folder Property Transfers: 163
    Total Number of Transfers: 343
    Number of Transfers Completed: 343
    Number of Transfers Failed: 0
    Number of Transfers Skipped: 0
    TotalBytesTransferred: 29703435
    Final Job Status: Completed
    
    PS C:\temp>
    
    

    There is a limitation for Azcopy to use on a network drive. When try to copy network drive with option  --preserve-smb-permissions=true, there is a panic runtime error message comes up. 

    Azcopy, Panic: runtime error: 



    PS C:\temp> ./azcopy.exe copy "J:\1992\" "https://fileshareca.file.core.windows.net/fs2/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2022-08-09T09%3A56%3A06Z&st=2022-08-09T01%3A56%3A06Z&spr=https&sig=C9Rikjasdfiowuerjkafi289389if89kj293%3D" --overwrite=prompt --from-to=LocalFile --follow-symlinks --put-md5 --follow-symlinks --preserve-smb-info=true --disable-auto-decoding=false --recursive --log-level=INFO
    INFO: Please note: the preserve-permissions flag is set to false, thus AzCopy will not copy SMB ACLs between the source and destination. To learn more: https://aka.ms/AzCopyandAzureFiles.
    INFO: Scanning...
    INFO: Any empty folders will be processed, because source and destination both support folders. For the same reason, properties defined on folders will be processed
    Job fb8eff17-2283-3248-63de-91dd1a3c5100 has started
    Log file is located at: C:\Users\jon\.azcopy\fb8eff17-2283-3248-63de-91dd1a3c5100.log
    INFO: Trying 4 concurrent connections (initial starting point)
    INFO: Trying 16 concurrent connections (seeking optimum)
    INFO: Trying 4 concurrent connections (backing off)
    INFO: Trying 8 concurrent connections (seeking optimum)
    76.4 %, 341 Done, 0 Failed, 2 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 12.0535 (Disk may be limiting spee80.3 %, 341 Done, 0 Failed, 2 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 13.1873 (Disk may be limiting speeINFO: Trying 4 concurrent connections (backing off)                                                                     
    80.3 %, 341 Done, 0 Failed, 2 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 13.1873 (Disk may be limiting spee94.4 %, 342 Done, 0 Failed, 1 Pending, 0 Skipped, 343 Total, 2-sec Throughput (Mb/s): 6.6049                            100.0 %, 343 Done, 0 Failed, 0 Pending, 0 Skipped, 343 Total,

    Job fb8eff17-2283-3248-63de-91dd1a3c5100 summary
    Elapsed Time (Minutes): 0.6026
    Number of File Transfers: 180
    Number of Folder Property Transfers: 163
    Total Number of Transfers: 343
    Number of Transfers Completed: 343
    Number of Transfers Failed: 0
    Number of Transfers Skipped: 0
    TotalBytesTransferred: 29703435
    Final Job Status: Completed
    PS C:\temp>
    PS C:\temp>
    PS C:\temp>
    PS C:\temp>
    PS C:\temp>  ./azcopy.exe copy "J:\1992\" "https://fileshareca.file.core.windows.net/fs2/Test/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2022-08-09T09:56:06Z&st=2022-08-09T01:56:06Z&spr=https&sig=C9Rz2jhdfkljasuweriqlkIUIUJHJIHIUUO*(^Y&&UUUUO78Q4%3D" --overwrite=prompt --from-to=LocalFile --follow-symlinks --put-md5 --follow-symlinks --preserve-smb-info=true --preserve-smb-permissions=true --disable-auto-decoding=false --recursive --log-level=INFO
    failed to parse user input due to error: cannot follow symlinks when preserving permissions (since the correct permission inheritance behaviour for symlink targets is undefined)
    PS C:\temp>  ./azcopy.exe copy "J:\1992\" "https://fileshareca.file.core.windows.net/fs2/Test/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2022-08-09T09:56:06Z&st=2022-08-09T01:56:06Z&spr=https&sig=C9jksdjkfioweuqjkhsdfiuqweiorjkasdhfiowui%3D" --overwrite=prompt --from-to=LocalFile --put-md5 --preserve-smb-info=true --preserve-smb-permissions=true --disable-auto-decoding=false --recursive --log-level=INFO
    INFO: Scanning...
    INFO: Any empty folders will be processed, because source and destination both support folders. For the same reason, properties and permissions defined on folders will be processed
    INFO: Trying 4 concurrent connections (initial starting point)
    Job 6318924a-e42c-8a49-67dd-b63adc7e3d37 has started
    Log file is located at: C:\Users\jon\.azcopy\6318924a-e42c-8a49-67dd-b63adc7e3d37.log
    panic: runtime error: index out of range [0] with length 0
    goroutine 94 [running]:
    github.com/Azure/azure-storage-azcopy/v10/ste.localFileSourceInfoProvider.GetSDDL({{0x1939bd0, 0xc0001729a0}, {{0x6318924a, 0xe42c, 0x8a49, {0x67, 0xdd, 0xb6, 0x3a, 0xdc, ...}}, ...}})
            /home/vsts/work/1/s/ste/sourceInfoProvider-Local_windows.go:78 +0x46c
    github.com/Azure/azure-storage-azcopy/v10/ste.(*azureFileSenderBase).addPermissionsToHeaders(_, {{0x6318924a, 0xe42c, 0x8a49, {0x67, 0xdd, 0xb6, 0x3a, 0xdc, 0x7e, ...}}, ...}, ...)
            /home/vsts/work/1/s/ste/sender-azureFile.go:274 +0x27a
    github.com/Azure/azure-storage-azcopy/v10/ste.(*azureFileSenderBase).Prologue(0xc00055ad80, {{0xc001fdc200, 0xc001fdc200, 0xc000516680}})
            /home/vsts/work/1/s/ste/sender-azureFile.go:154 +0x358
    github.com/Azure/azure-storage-azcopy/v10/ste.scheduleSendChunks({0x1939bd0, 0xc0001729a0}, {0xc0005ee000, 0x1b}, {0x19110f0, 0xc00000a8e8}, 0x4000, {0x1923940, 0xc00055ad80}, 0xc001520990, ...)
            /home/vsts/work/1/s/ste/xfer-anyToRemote-file.go:430 +0x6d2
    github.com/Azure/azure-storage-azcopy/v10/ste.anyToRemote_file({_, _}, {{0x6318924a, 0xe42c, 0x8a49, {0x67, 0xdd, 0xb6, 0x3a, 0xdc, ...}}, ...}, ...)
            /home/vsts/work/1/s/ste/xfer-anyToRemote-file.go:347 +0xe27
    github.com/Azure/azure-storage-azcopy/v10/ste.anyToRemote({0x1939bd0, 0xc0001729a0}, {0x18ff740, 0xc000510000}, {0x1e0ff01fe28, 0xc000510240}, 0xc0005f2ac0, 0x2d)
            /home/vsts/work/1/s/ste/xfer-anyToRemote-file.go:184 +0x37d
    github.com/Azure/azure-storage-azcopy/v10/ste.parameterizeSend.func1({0x1939bd0, 0xc0001729a0}, {0x18ff740, 0xc000510000}, {0x1e0ff01fe28, 0xc000510240})
            /home/vsts/work/1/s/ste/xfer.go:77 +0x44
    github.com/Azure/azure-storage-azcopy/v10/ste.(*jobPartMgr).StartJobXfer(0x1f9b6a0, {0x1939bd0, 0xc0001729a0})
            /home/vsts/work/1/s/ste/mgr-JobPartMgr.go:712 +0x50
    github.com/Azure/azure-storage-azcopy/v10/ste.(*jobPartTransferMgr).StartJobXfer(0xc0001729a0)
            /home/vsts/work/1/s/ste/mgr-JobPartTransferMgr.go:239 +0x32
    github.com/Azure/azure-storage-azcopy/v10/ste.(*jobMgr).transferProcessor.func1({0x1939bd0, 0xc0001729a0})
            /home/vsts/work/1/s/ste/mgr-JobMgr.go:1015 +0x1b9
    github.com/Azure/azure-storage-azcopy/v10/ste.(*jobMgr).transferProcessor(0xc000395880, 0x0)
            /home/vsts/work/1/s/ste/mgr-JobMgr.go:1027 +0xc2
    created by github.com/Azure/azure-storage-azcopy/v10/ste.NewJobMgr
            /home/vsts/work/1/s/ste/mgr-JobMgr.go:200 +0xbf4
    PS C:\temp>


    Since J drive is network share drive which was manually mapped using File Explorer, Azcopy tool does not support it well. But if you change source drive from network drive to local, the command will run well. All SMB permissions will be copied over to remote Azure storage file share. 

    Notes:

    Permissions on a shared resource, such as a folder or volume, are determined by the local NTFS permissions for that resource and by the protocol used to access the shared resource:

    • Server Message Block (SMB) protocol

      SMB–based access control (for Windows-based file systems) is implemented by granting permissions to the individual users and groups.

    • Network File System (NFS) protocol

      NFS–based access control (for UNIX-based file systems) is implemented by granting permissions to specific client computers and groups, using network names.
    The final access permissions for a shared resource are determined by considering both the NTFS permissions and the sharing protocol permissions, and then applying the more restrictive permissions. If you enable access-based enumeration on an SMB-based shared folder, Windows hides files and folders for which users do not have Read permissions.

    Fix issue (TBT):

    MS Doc:

    So found a way to get it to work. Instead of mapping the driving using the net use command I used the script provided from the Azure portal for that share. You can get this from clicking on the share in the portal and then the connect button it will give you two options for Authentication AD or Storage Key. I selected storage key and copied the script and ran that in powershell. I was then able to copy ntfs permissions with robocopy or azcopy. 


    Command:

    ./azcopy.exe copy "C:\Test\1991\" "https://fileshares.file.core.windows.net/fs2/Test5/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2022-08-23T02:26:57Z&st=2022-08-22T18:26:57Z&spr=https&sig=aFpvadfceMrDVdfsder45ERTrfgdE%2BZRY%2BmA%3D" --from-to=LocalFile --preserve-smb-info=true --preserve-smb-permissions=true --recursive
    


    > ROBOCOPY /Mir <Source> <Target>
    > ROBOCOPY /E /Copy:S /IS /IT <Source> <Target>


    The first Robocopy command above will copy data and security for files that have been updated, and the second Robocopy command will refresh file security for all files, without copying any file data.

    robocopy /Mir J:\source_folder O:\destination_folder\test


    ROBOCOPY "\\Server\Share" "\\Server\Share" /E /SEC /COPY:DT /R:1 /W:1 /DCOPY:T /V /TEE /log:"C:\Users\Username\Documents\Share.log"

    Generate AzCopy Command from Azure Storage Explorer

     
    Start an upload task, then you will find an option to "Copy AzCopy Command to Clipboard"


    Generated command will looks like this:

    $env:AZCOPY_CRED_TYPE = "Anonymous";

    ./azcopy.exe copy "C:\test\1992\" "https://fileshareca.file.core.windows.net/fs2/?sv=2021-04-10&se=2022-09-08T13%3A43%3A24Z&sr=s&sp=rwl&sig=G16mpJRMeajshfjJHJHGH78384sjhfjYIUI%3D" --overwrite=prompt --from-to=LocalFile --follow-symlinks --put-md5 --follow-symlinks --preserve-smb-info=true --disable-auto-decoding=false --recursive --log-level=INFO;

    $env:AZCOPY_CRED_TYPE = "";






    Troubleshooting

    By default, the log and plan files are located in the %USERPROFILE%\.azcopy directory on Windows or $HOME$\.azcopy directory on Mac and Linux, but you can change that location.

    Folder's timestamp will not be kept but files will be. 


    If you are facing some kind of SSL/TLS 1.2 support issue, you might want to change your OS to support TLS 1.2.


    PS C:\Windows\system32> iwr https://onegetcdn.azureedge.net/providers/nuget-2.8.5.208.package.swidtag
    iwr : The request was aborted: Could not create SSL/TLS secure channel.
    At line:1 char:1
    + iwr https://onegetcdn.azureedge.net/providers/nuget-2.8.5.208.package ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
       eption
        + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
    
    PS C:\Windows\system32> [Enum]::GetNames([Net.SecurityProtocolType]) -contains 'Tls12'
    True
    PS C:\Windows\system32> [System.Net.ServicePointManager]::SecurityProtocol.HasFlag([Net.SecurityProtocolType]::Tls12)
    False
    PS C:\Windows\system32>
    PS C:\Windows\system32> [Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [
    Net.SecurityProtocolType]::Tls12
    PS C:\Windows\system32> [System.Net.ServicePointManager]::SecurityProtocol.HasFlag([Net.SecurityProtocolType]::Tls12)
    True
    PS C:\Windows\system32> iwr https://onegetcdn.azureedge.net/providers/nuget-2.8.5.208.package.swidtag
    
    
    StatusCode        : 200
    StatusDescription : OK
    Content           : <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
                        <SoftwareIdentity xmlns="http://standards.iso.org/iso/19770/-2/2015/schema.xsd"
                        xmlns:discovery="http://packagemanagement.org/discovery" xmlns...
    RawContent        : HTTP/1.1 200 OK
                        Age: 98079
                        Content-MD5: JsUBlau/3mYRpMruNYWWCw==
                        Vary: Accept-Encoding
                        X-Cache: HIT
                        x-ms-blob-type: BlockBlob
                        x-ms-lease-status: unlocked
                        x-ms-request-id: 830572a7-801e-0035-5c4...
    Forms             : {}
    Headers           : {[Age, 98079], [Content-MD5, JsUBlau/3mYRpMruNYWWCw==], [Vary, Accept-Encoding], [X-Cache, HIT]...}
    Images            : {}
    InputFields       : {}
    Links             : {}
    ParsedHtml        : System.__ComObject
    RawContentLength  : 822
    


    Or Registry Key


    Windows Registry Editor Version 5.00
    
    [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319]
    "SchUseStrongCrypto"=dword:00000001
    
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319]
    "SchUseStrongCrypto"=dword:00000001
    




    3 comments:

    1. The best way to Migrate Local Files to Azure blob Storage with SMB Permission is by using GUI backup/sync tools like Gs Richcopy 360 or Sharegate

      ReplyDelete
      Replies
      1. Hi Hansil
        Are ShareGate and Gs Richcopy360 supports SMB?

        Delete
    2. Thanks Hansil for those suggested tools. I haven't got a chance to use them, but both seems are very promising in their websites.

      ReplyDelete