🦩 Exploring MinIO – High Performance Standalone Object Storage S3 Compatible

Are you looking for object storage software to host on your server?

There are many cloud object stores like AWS S3, etc.

However, if you need to host the data on your server, MinIO can help with this task.

MiniIO is a popular open source distributed object storage software S3 compatible.

It is known for its high performance.

You can use MinIO for simple web applications to large data distribution workloads for analytics and machine learning applications.

Pros of using:

  • Standard flat file storage
  • Distributing data across multiple clouds
  • Disaster recovery
  • Data analytics

Is it bulky software?

No, it weighs 50 MB and is suitable for Kubernetes.

Minio writes data and metadata as an object.

This removes the reliance on having additional database or software to store metadata and improve performance.

Architecture from their official website:

🦩 Exploring MinIO - High Performance Standalone Object Storage S3 Compatible

Let’s take a look at some of the features worth paying attention to.

  • High performance! Minio is capable of reading / writing at ~ 170GB / s. It’s a lot!
  • Scalability – use clustering and scale as needed
  • Cloud-native
  • Data protection using the Erasure code method
  • Multiple encryption supported, including AES-CBC, AES-256-GCM, ChaCha20
  • Compatible with regular KMS
  • Event notification
  • Compatible with etcd and CoreDNS

MinIO is a good choice for software-distributed storage. Let’s see how to set it up.

Installing the MinIO server

You can install it on Linux, Windows, macOS, and via Kubernetes.

Prefer to build from source?

Of course you can, if you have Golang installed.

For this article, I’ll install it on CentOS!

  • Login to server
  • Create a folder on the desired file system. For example minio-server
  • Go to the newly created folder and run the below wget command.
wget https://dl.min.io/server/minio/release/linux-amd64/minio

The system will load the binary and this file should look like this:

-rw-r--r--  1 root root 48271360 Oct 18 21:57 minio

Make the file executable with chmod command

chmod 755 minio

Let’s run MinIO as a server.

./minio server /data &

/ data mentioned above is the filesystem that MinIO will store objects in.

Startup is fast and you should see information like this:

Endpoint:  http://xx.71.141.xx:9000 http://127.0.0.1:9000
AccessKey: minioadmin
SecretKey: minioadmin

Browser Access:
   http://xx.71.141.xx:9000 http://127.0.0.1:9000

Command-line Access: https://docs.min.io/docs/minio-client-quickstart-guide
   $ mc alias set myminio http://xx.71.141.xx:9000 minioadmin minioadmin

Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
Detected default credentials 'minioadmin:minioadmin', please change the credentials immediately using 'MINIO_ACCESS_KEY' and 'MINIO_SECRET_KEY'

Let’s login to MinIO via browser with default credentials – minioadmin: minioadmin

The interface is very neat and simple, but first of all, let’s change the default credentials as this puts the risk of tampering. To change the default MinIO credentials, we will export the access key and private key as shown below and run MinIO.

export MINIO_ACCESS_KEY=itsecforu
export MINIO_SECRET_KEY=itsecpassword
./minio server /data &

Now it shouldn’t complain and issue a warning about the detection of default credentials.

Let’s try to upload files.

  • Click the + icon in the bottom right corner and create a bucket
  • I have uploaded a test file and it is immediately visible in the browser

🦩 Exploring MinIO - High Performance Standalone Object Storage S3 Compatible

and on the server:

ls -ltr
total 4
-rw-r--r-- 1 root root 11 Oct 19 11:09 MinIO-Test.txt

If you click the share button on a file in your browser, you will receive a share link and an option to set an expiration date.

MinIO client

The MinIO client is more than just aws-cli that allows you to manage storage.

The client is available for Windows, macOS and Linux.

To install on Linux, run the following:

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod 755 mc

Run the mc command to see the command help.

/mc
NAME:
  mc - MinIO Client for cloud storage and filesystems.

USAGE:
  mc [FLAGS] COMMAND [COMMAND FLAGS | -h] [ARGUMENTS...]

COMMANDS:
  alias      set, remove and list aliases in configuration file
  ls         list buckets and objects
  mb         make a bucket
  rb         remove a bucket
  cp         copy objects
  mirror     synchronize object(s) to a remote site
  cat        display object contents
  head       display first 'n' lines of an object
  pipe       stream STDIN to an object
  share      generate URL for temporary access to an object
  find       search for objects
  sql        run sql queries on objects
  stat       show object metadata
  mv         move objects
  tree       list buckets and objects in a tree format
  du         summarize disk usage recursively
  retention  set retention for object(s)
  legalhold  manage legal hold for object(s)
  diff       list differences in object name, size, and date between two buckets
  rm         remove objects
  version    manage bucket versioning
  ilm        manage bucket lifecycle
  encrypt    manage bucket encryption config
  event      manage object notifications
  watch      listen for object notification events
  undo       undo PUT/DELETE operations
  policy     manage anonymous access to buckets and objects
  tag        manage tags for bucket and object(s)
  replicate  configure server side bucket replication
  admin      manage MinIO servers
  update     update mc to latest release

GLOBAL FLAGS:
  --autocompletion              install auto-completion for your shell
  --config-dir value, -C value  path to configuration folder (default: "/root/.mc")
  --quiet, -q                   disable progress bar display
  --no-color                    disable color theme
  --json                        enable JSON lines formatted output
  --debug                       enable debug output
  --insecure                    disable SSL certificate verification
  --help, -h                    show help
  --version, -v                 print the version

TIP:
  Use 'mc --autocompletion' to enable shell autocompletion

VERSION:
  RELEASE.2020-10-03T02-54-56Z

Let’s try to find the file that I downloaded using the mc command.

First, we need to set the alias for the repository we want to manage.

/mc alias set minio http://xx.71.141.xx:9000/ itsecforu itsecpassword
Added `minio` successfully.
  • minio is an alias. You can change it to any value.
  • Change the HTTP url to yours.
  • Change the access key and private key to your own.

Great, everything works!

Literally everything can be done through the client.

You can manage not only MinIO cloud storage, but also GCS, AWS S3, Azure.

MinIO SDK

Depending on the stack of your application, you can programmatically interact with the object store using the SDK.

It supports Go, Python, Node.js, .NET, Haskell, and Java.

MinIO Gateway

Add MiniIO gateway to S3, Azure, NAS, HDFS to take advantage of MinIO browser and disk caching.

Conclusion

Whether you’re looking for private, hybrid, or multi-cloud object storage, MinIO looks promising.

Sidebar