67228: Using archive_ctl CLI tool to test upload and download speed to/from a cloud storage location

use Google Translate

Last update: Wed, 2021-02-24 18:48

Applies to both Acronis-hosted and partner-hosted cloud locations.

Preparation

1. Download archive_io_ctl, unpack and place it on a machine in your environment.

2. Find out the cloud storage address from the Management Portal web GUI:

3. Obtain the cloud certificate from a currently registered and active (which has recently done successful backups) Agent:

  1. See https://kb.acronis.com/content/60082 for certificate locations.
  2. Rename the file into cert.crt. 
  3. Put the cert.crt file in the same directory where archive_io_ctl is.

4. Create a file named test.bin and having the size of 1GB:

  1. Open an elevated Command prompt (in the search box on the taskbar, type cmd, right-click Command prompt and select Run as administrator).
  2. Navigate to the directory where you saved archive_io_ctl.
  3. Run this command:
    fsutil file createnew test.bin 1073741824

Testing Upload Speed

To test upload speed, use an existing cloud storage (allows to measure the effective throughput of the network between the test machine and the cloud location plus sequential write performance of the cloud storage; this does not take into account source-side disk-reading overheads).

  1. Download the script and place it to the archive_io_ctl folder.
  2. Open an elevated Command prompt.
  3. Use the cd command to navigate to the directory where acrhive_io_ctl has been unpacked. For example, you have unpacked it to C:\archive_io_ctl, then execute:
    cd C:\archive_io_ctl

  4. Execute the script:
    run_test.bat <cloud_storage_address>
    where <cloud_storage_address> is the address you've noted in Preparation - Step 2
    For example:
    run_test.bat baas-fes-eu4.acronis.com

"End copy with 0" means that the script ran successfully. You can see the upload time:

Testing Download Speed

To test the download speed, use an existing file (such as a TIBX backup archive or a file you have uploaded in "Testing Upload Speed") in the cloud storage and a local machine (allows to measure the effective throughput of the network between the test machine and the cloud location plus sequential read performance of the cloud storage; this does not take into account local-machine-side disk-writing overheads).

1. To list files in the cloud repository, use this sequence of commands:

To connect to your cloud storage, execute:
archive_io_ctl --astor <storage_address> --cert cert.crt
where <storage_address> is the address you've noted in Preparation - Step 2;
cert.crt is your cloud certificate file obtained in Preparation - Step 3

An interactive prompt/shell will appear once archive_io_ctl has connected and authenticated with the cloud storage.

To navigate to the folder where the archives or the previously uploaded file is located, type:
cd 1

To list the contents of this folder, type:
ls

In the list of files and their logical and occupied on-disk sizes, you will see the previously uploaded file.

Type exit to exit the interactive shell.

2. Note the time(stamp) at download start. The archive_io_ctl tool does print timestamps, however, too much text can get printed to the CMD terminal, causing its text buffer to overflow, losing the first lines thus the start_time timestamp.

3. Download a file from Acronis cloud storage with this command:

archive_io_ctl --astor <storage_address> --cert cert.crt --copy /1/test.bin --dst <local_path_and_filename_with_extension> --continue

where <storage_address> is the address you've noted in Preparation - Step 2
cert.bin is your cloud certificate file obtained in Preparation - Step 3
test.bin is the file you want to download, noted in Testing Download Speed - Step 1
<local_path_and_filename_with_extension> is the path to the folder where this file should be saved and the filename it should be saved with

For example:
archive_io_ctl --astor baas-fes-eu4.acronis.com --cert cert.crt --copy /1/test.bin --dst C:\archive_io_ctl\newtest.bin --continue

3. After the download finishes, note the time(stamp). Subtract the start time from the end time and divide the uploaded file size by the time spent; the result is the measured average download speed.

Deleting test file from cloud 

Delete the uploaded test file so that it doesn't consume cloud storage quota unnecessarily:

archive_io_ctl --astor <storage_address> --cert cert.crt --rm /1/test.bin

where <storage_address> is the address you've noted in Preparation - Step 2
<certificate file name> is your cloud certificate file obtained in Preparation - Step 3
test.bin is the file that you downloaded for testing purposes in Testing Download Speed

More information

It is generally a good idea to repeat these tests a few times, during different times of day and different days of week, to get a better idea of what results to expect as the load on the local network, the network path(s) local machine <-> cloud storage, and the I/O load on the cloud storage vary with time of day and day of week (network load and storage load).

NOTE: The transfer done by archive_io_ctl is, for the time being, over a single TCP stream, which matches how most agents work with cloud storage of most tenants at the moment. The currently used storage access protocol, ABGW 1.x, does not use multiple streams in parallel. A next-gen ABGW2 protocol is undergoing testing and should gradually be rolled out next year.

NOTE: The speedtest tool which is described in KB https://kb.acronis.com/content/59690 uses multiple TCP streams (as of now, typically 4) in parallel. This is less realistic and is in many cases not representative of how backups are transferred to/from cloud; if the latency between the testing machine/agent and the cloud storage exceeds several tens of milliseconds, the speedtest application is likely to SUBSTANTIALLY OVERESTIMATE THE TRANSFER SPEED. Please also note that the speedtest web application cannot be used with partner-hosted storage (storage which is not in an Acronis datacenter). The described method using archive_io_ctl is however UNIVERSALLY APPLICABLE.

NOTE: The obtained upload and download speeds should be used only as rough estimates of the maximum transfer speeds to/from cloud; the processing of real backups is usually somewhat slower (e.g. 20-30% slower is normal). If your actual backup transfer speeds are much slower than that, check for I/O and CPU bottlenecks (saturation) of the local machine on which you're doing the tests.

Tags: