Download all blobs files locally from azure container using python

Question

I'm using python 3.6 in visual studio and I want to download all blobs files from my azure container in single folder. This is my code but the problem is, that it downloads 1 blob file in the folder and then when downloading the second file it overwrite the first file and in the end I only have the last blob in my local folder. How can I download all the blobs files at once in a single folder?

from azure.storage.blob import BlockBlobService
block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)

generator = block_blob_service.list_blobs(CONTAINER_NAME)
        for blob in generator:
            block_blob_service.get_blob_to_path(CONTAINER_NAME, blob.name, LOCAL_FILE)

Do you mean to download them and append the blobs into a single file? — Hai Vu
– Hai Vu, Commented Mar 18, 2017 at 14:51
You'd need to combine them yourself. Or maybe download them to a single stream. But the get_blob_to_path() method isn't going to append to files. — David Makogon
– David Makogon, Commented Mar 18, 2017 at 15:03
Have you tried to pass open_mode='ab' to get_blob_to_path? — Hai Vu
– Hai Vu, Commented Mar 18, 2017 at 15:03

Peter Pan · Accepted Answer · 2017-03-20 06:34:53Z

16

Based on my understanding, I think there are two solutions for your needs.

Download all blobs from a container, and write these blob content to a single file via the method get_blob_to_bytes or get_blob_to_stream, please see my sample code as below.

from azure.storage.blob import BlockBlobService

block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)

generator = block_blob_service.list_blobs(CONTAINER_NAME)

fp = open('<your-local-file-name>', 'ab')

for blob in generator:
    # Using `get_blob_to_bytes`
    b = service.get_blob_to_bytes(container_name, blob.name)
    fp.write(b.content)
    # Or using `get_blob_to_stream`
    # service.get_blob_to_stream(container_name, blob.name, fp)

fp.flush()
fp.close()

Download all blobs from a container, and write these blobs into a zip file via the method get_blob_to_bytes, please see my sample code below.

from azure.storage.blob import BlockBlobService
import zipfile

block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)

generator = block_blob_service.list_blobs(CONTAINER_NAME)

zf = zipfile.ZipFile(CONTAINER_NAME+'.zip', 
             mode='w',
             compression=zipfile.ZIP_DEFLATED, 
             )

for blob in generator:
    b = service.get_blob_to_bytes(container_name, blob.name)
    zf.writestr(blob.name, b.content)

zf.close()

Hope it helps. Any concern, please feel free to let me know.

answered Mar 20, 2017 at 6:34

Peter Pan

24.2k4 gold badges31 silver badges47 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Priyansh Over a year ago

Perfect answer..!! This is what I was looking. I think question should be edited so that google can index it.

Lakshika Parihar · Accepted Answer · 2022-02-02 06:37:43Z

1

In the azure-storage-blob version 12.9.0 , I used the below script :

        from azure.storage.blob import BlobServiceClient
        import zipfile
        conn_str = ''
        block_blob_service = BlobServiceClient.from_connection_string(conn_str)
        generator = block_blob_service.get_container_client(container_name)
        my_blobs = generator.list_blobs()

        zf = zipfile.ZipFile("data"+'.zip', mode='w', compression=zipfile.ZIP_DEFLATED)

        for blob in my_blobs:
                bytes = generator.get_blob_client(blob.name).download_blob().readall()
                zf.writestr(blob.name, bytes)

        zf.close()

answered Feb 2, 2022 at 6:37

Lakshika Parihar

1,18311 silver badges19 bronze badges

Collectives™ on Stack Overflow

Download all blobs files locally from azure container using python

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related