minio/minio-py

MinIO Python Client SDK for Amazon S3 Compatible Cloud Storage

The MinIO Python Client SDK provides high level APIs to access any MinIO Object Storage or other Amazon S3 compatible service.

This Quickstart Guide covers how to install the MinIO client SDK, connect to the object storage service, and create a sample file uploader.

The example below uses:

• Python version 3.10+
• The MinIO mc command line tool
• The MinIO play test server

The play server is a public MinIO cluster located at https://play.min.io. This cluster runs the latest stable version of MinIO and may be used for testing and development. The access credentials in the example are open to the public and all data uploaded to play should be considered public and world-readable.

For a complete list of APIs and examples, see the Python SDK Documentation

Install the MinIO Python SDK

The Python SDK requires Python version 3.10+. You can install the SDK with pip or from the minio/minio-py GitHub repository:

Using pip

pip3 install minio

Using Source From GitHub

git clone https://github.com/minio/minio-py
cd minio-py
python setup.py install

Create a MinIO Client

To connect to the target service, create a MinIO client using the Minio() method with the following required parameters:

For example:

from minio import Minio

client = Minio(
    endpoint="play.min.io",
    access_key="Q3AM3UQ867SPQQA43P2F",
    secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG",
)

Example - File Uploader

This example does the following:

• Connects to the MinIO play server using the provided credentials.
• Creates a bucket named python-test-bucket if it does not already exist.
• Uploads a file named test-file.txt from /tmp, renaming it my-test-file.txt.
• Verifies the file was created using mc ls.

file_uploader.py

# file_uploader.py MinIO Python SDK example
from minio import Minio
from minio.error import S3Error

def main():
    # Create a client with the MinIO server playground, its access key
    # and secret key.
    client = Minio(
        endpoint="play.min.io",
        access_key="Q3AM3UQ867SPQQA43P2F",
        secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG",
    )

    # The file to upload, change this path if needed
    source_file = "/tmp/test-file.txt"

    # The destination bucket and filename on the MinIO server
    bucket_name = "python-test-bucket"
    destination_file = "my-test-file.txt"
    
    # Make the bucket if it doesn't exist.
    found = client.bucket_exists(bucket_name=bucket_name)
    if not found:
        client.make_bucket(bucket_name=bucket_name)
        print("Created bucket", bucket_name)
    else:
        print("Bucket", bucket_name, "already exists")

    # Upload the file, renaming it in the process
    client.fput_object(
        bucket_name=bucket_name,
        object_name=destination_file,
        file_path=source_file,
    )
    print(
        source_file, "successfully uploaded as object",
        destination_file, "to bucket", bucket_name,
    )

if __name__ == "__main__":
    try:
        main()
    except S3Error as exc:
        print("error occurred.", exc)

To run this example:

1. Create a file in /tmp named test-file.txt. To use a different path or filename, modify the value of source_file.

2. Run file_uploader.py with the following command:

python file_uploader.py

If the bucket does not exist on the server, the output resembles the following:

Created bucket python-test-bucket
/tmp/test-file.txt successfully uploaded as object my-test-file.txt to bucket python-test-bucket

3. Verify the uploaded file with mc ls:

mc ls play/python-test-bucket
[2023-11-03 22:18:54 UTC]  20KiB STANDARD my-test-file.txt

RDMA (optional)

put_object and get_object can dispatch to MinIO's RDMA path via minio-cpp. It is strictly opt-in and the SDK stays pure-Python unless used.

from minio import Minio

client = Minio(
    endpoint="server:9000",
    access_key="...",
    secret_key="...",
    secure=False,
    enable_rdma=True,             # opt-in
)

buf = bytearray(1 << 20)
client.put_object(
    bucket_name="b", object_name="o",
    data=buf, length=len(buf),    # buffer-protocol object selects RDMA
)

dst = bytearray(1 << 20)
n = client.get_object(
    bucket_name="b", object_name="o",
    into=dst, length=len(dst),    # into= selects RDMA, returns bytes
)

Requires libminiocpp.so (built with -DMINIO_CPP_ENABLE_RDMA=ON) on the host's library search path (or pointed at via the MINIOCPP_LIB env var). It carries its own RDMA transport, libs3rdma, which supports RoCE and native InfiniBand and needs no NVIDIA client library on the host.

GPU buffer pointers (e.g. CuPy's arr.data.ptr, PyTorch's t.data_ptr()) work as data= / into= arguments unchanged — pass them as int. The SDK links no CUDA itself; allocating device memory stays the application's job.

Set S3RDMA_DEVICE to pin a particular HCA on a multi-NIC host; otherwise the first usable device is selected.

See examples/put_object_rdma.py and examples/get_object_rdma.py.

More References

• Python SDK Documentation
• Examples

Explore Further

• Complete Documentation

Contribute

Contributors Guide

License

This SDK is distributed under the Apache License, Version 2.0, see LICENSE and NOTICE for more information.