Find Nuclei
Posts Tags Categories About
Find Nuclei

Contents

MinIO Gateway with DigitalOcean Spaces, NGINX and auth0

Emil Rozbicki included in Technology
   2423 words   12 minutes 

Intro

This post illustrates how to install MinIO Gateway for DigitalOcean Spaces. There’s lots of great guides for how to set up all the necessary components already so where possible the links to these resources will be provided. For the authentication and authorization we’ll use a free Auth0 account. The fully self-hosted solution is also possible via KeyCloak or similar open source tools.

Tech stack

Cost

Cost of the implementation per month: USD $10 + transfer charge above the droplet and storage limits (so far haven’t managed to go over the limits).

Installation includes provisioning a number of resources with different providers, including: DigitalOcean, auth0 and GoDaddy. The cost of the domain for the 1st year is negligible and the cost of the DigitalOcean Droplet (s-1vcpu-1gb) with Spaces is around USD $10 per month:

Droplet cost breakdown

DigitalOcean instance

DigitalOcean is a good alternative for AWS or Azure when it comes to testing things out. Very simple pricing structure, easy to work with management console make it a very attractive environment for the development projects like this.

Few easy steps to get things going:

  1. Sing up, if you haven’t already.
  2. Create a new Project for the MinIO resources

    DigitalOcean new Project

  3. Provision a new droplet or use an existing one. The rest of the walk-through is based on Ubuntu 20.04. The provisioning process is very straight forward. Choose the right data center region, add monitoring, public IP address, SSH keys and assign it to the right Project.

    DigitalOcean new Droplet

  4. Provision Spaces object store, S3 in DO-land.

With all this we should be set for the next steps, basic NGINX configuration and MinIO install.

Custom Domain

To enable oAuth authentication with 3rd party provider a custom domain is required. Below is example DNS configuration for a domain registered with GoDaddy.

GoDaddy DNS config.

Note: Above config already contains minio and minio-server CNAME that will be later used to configure NGINX for MinIO console and MinIO server respectively.

NGINX configuration

  1. Follow instructions to install NGINX. If you run everything correctly you should see at droplets IP address:

    NGINX home page

  2. Use Certbot to provision and add SSL cert for the domain registered in Custom Domain step to NGINX configuration.
    1. DigitalOcean provides a great guide on how to do this
  3. After installation is complete the NGINX config should look very similar to the following:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

server {
    # Server a static website from /var/www
    root /var/www/domain-name.com/html;
    index index.html index.htm index.nginx-debian.html;

    server_name domain-name.com www.domain-name.com;

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
  
    # HTTP redirects to HTTPS
    if ($host = www.domain-name.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = domain-name.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen 80;
    listen [::]:80;

    server_name domain-name.com www.domain-name.com;
    return 404; # managed by Certbot

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
}

MinIO Gateway installation

  1. There’s an existing installation guide MinIO Server installation for DigitalOcean here.
    1. Follow the installation Step 1 and Step 2.
    2. We’ll use NGINX to proxy_pass our custom minio subdomains to MinIO server and MinIO console.
    3. SSL certs can be provisioned with certbot just like for the full domain in the previous step.
  2. Adjust config from the DigitalOcean MinIO guide:
    1. fix the server and console ports to 9000 and 9001 respectively
    2. Add DigitalOcean Spaces Endpoint - you can find the correct value in the Spaces settings
    3. MINIO_ROOT_USER and MINIO_ROOT_PASSWORD are the SECRET and KEY for the DigitalOcean Spaces.
1
2
3
4
5
6
7

root@server:~# cat /etc/default/minio
MINIO_VOLUMES="/usr/local/share/minio/"
MINIO_OPTS="-C /etc/minio --address :9000 --console-address :9001"
MINIO_ROOT_USER="*********"
MINIO_ROOT_PASSWORD="*********"
MINIO_BROWSER_REDIRECT_URL="https://minio.domain-name.com"
DIGITALOCEAN_ENDPOINT="Check Spaces settings for end point"
  1. systemd service file adjusted for MinIO Gateway:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

 root@server:~# cat /etc/systemd/system/minio-gateway.service
[Unit]
Description=MinIO Gateway
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/usr/local/

User=minio-user
Group=minio-user
ProtectProc=invisible

EnvironmentFile=/etc/default/minio
ExecStart=/usr/local/bin/minio gateway s3 $DIGITALOCEAN_ENDPOINT $MINIO_OPTS

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=1048576

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target
  1. The MinIO Gateway and Console should start at this point but won’t be reachable since we do not have NGINX config yet.
  2. If you haven’t added subdomains for the server and console, yet you’ll need to add it to the DNS record, e.g.:
    1. minio.domain-name.com
    2. minio-server.domain-name.com
  3. Run certbot to get certs for new subdomains, only pass the domains and subdomains you’re going to use:
1

certbot --nginx -d domain-name.com -d www.domain-name.com -d minio.domain-name.com -d minio-server.domain-name.com

Check this guide out for more details

  1. certbot should update the NGINX configuration automatically. You can also do it yourself, it should look similar to:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

 server {
    # MinIO console
    server_name minio.domain-name.com;
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    # MinIO server
    server_name minio-server.domain-name.com;
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    # Static site
    root /var/www/domain-name.com/html;
    index index.html index.htm index.nginx-debian.html;

    server_name domain-name.com www.domain-name.com;

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = minio.domain-name.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    if ($host = minio-server.domain-name.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    if ($host = www.domain-name.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = domain-name.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen 80;
    listen [::]:80;

    server_name domain-name.com www.domain-name.com minio.domain-name.com minio-server.domain-name.com;
    return 404; # managed by Certbot

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
  1. NGINX config for MinIO Server API and MinIO Console. We’ll use the info from the MinIO Cookbook to set the appropriate config for the server and console:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

 server {
    # MinIO console
    server_name minio.domain-name.com;
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
    
    # From https://docs.min.io/docs/setup-nginx-proxy-with-minio.html
    # To allow special characters in headers
    ignore_invalid_headers off;
    # Allow any size file to be uploaded.
    # Set to a value such as 1000m; to restrict file size to a specific value
    client_max_body_size 0;
    # To disable buffering
    proxy_buffering off;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        proxy_set_header Host $host;
        proxy_connect_timeout 300;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        chunked_transfer_encoding off;

        proxy_pass http://localhost:9001;
   }
}

server {
    # MinIO server
    server_name minio-server.domain-name.com;
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/domain-name.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/domain-name.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
    
    # From https://docs.min.io/docs/setup-nginx-proxy-with-minio.html
    # To allow special characters in headers
    ignore_invalid_headers off;
    # Allow any size file to be uploaded.
    # Set to a value such as 1000m; to restrict file size to a specific value
    client_max_body_size 0;
    # To disable buffering
    proxy_buffering off;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        proxy_set_header Host $host;
        proxy_connect_timeout 300;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        chunked_transfer_encoding off;

        proxy_pass http://localhost:9000;
   }

}

At this point you should have MinIO console accessible at minio.domain-name.com. Since you only have a single user for now you’ll need to use the MinIO Root user login and password to login. In the next step we’ll configure MinIO to use Auth0 as authentication provider.

If things are not running check the debugging section to find out how to get some insight to what’s happening with your MinIO.

Auth0.com

Auth0 provides a great flexibility and tooling around Single SingOn authorization and authentication services which comes very handy when building new services.

  1. Create a free account.
  2. Create a new application from the Applications menu. In this example the application will be called minio-domain-name.
    1. Choose a single page application template.
    2. In the settings page add the following information:
      1. Application Login URI: https://minio.domain-name.com/login
      2. Allowed Callback URLs: https://minio.domain-name.com/oauth_callback
    3. The above settings will allow the Auth0 to authenticate and authorize the user to MinIO Console via Single Sign On.
  3. Create a new User from the User Management menu. Add user to the minio-domain-name in the Authorized Applications tab.
  4. MinIO expects policy claims to be available in the id token. Since policy claims are not part of the standard we’ll need to use Auth Pipeline Rule to add MinIO policy claims to the id tokens.
    1. Create a new Rule from the Auth Pipeline -> Rules menu.
    2. There’s lots of potential ways of sorting out the authorization and access policies to the buckets, external DBs, Auth0 API permissions, etc. In this simple example all registered users will receive the readwrite permissions. In the Script input of the new Rule paste the following:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

    function (user, context, callback) {
  const namespace = 'https://minio';

  let idTokenClaims = context.idToken || {};
  let accessTokenClaims = context.accessToken || {};

  idTokenClaims[`${namespace}/policy`] = "readwrite";
  accessTokenClaims[`${namespace}/policy`] = "readwrite";

  context.idToken = idTokenClaims;
  context.accessToken = accessTokenClaims;
  callback(null, user, context);
}
    1. You can check the existing policies with MinIO client by running:
    1
2
3
4
5
6

    mc admin policy list local
  readwrite
  writeonly
  consoleAdmin
  diagnostics
  readonly
  5. Advance authorization with roles. Following the built-in MinIO policies we can build a simple authorization system with Auth0 by using Roles.
    1. Adding MinIO roles to Auth0:

      MinIO -> Auth0

    2. Adjust the Auth Pipeline Rule to use the new policy mechanism:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

    function (user, context, callback) {
  const namespace = 'https://minio';

  const assignedRoles = (context.authorization || {}).roles;

  let idTokenClaims = context.idToken || {};
  let accessTokenClaims = context.accessToken || {};

  idTokenClaims[`${namespace}/policy`] = assignedRoles;
  accessTokenClaims[`${namespace}/policy`] = assignedRoles;

  context.idToken = idTokenClaims;
  context.accessToken = accessTokenClaims;
  callback(null, user, context);
}
  6. Add appropriate roles to User settings in Auth0. Changing the roles should be reflected in MinIO after next login.
  7. Adjust MinIO config to use Single Sing On.
    1. Client ID and Client Secret are available from the Application, Settings menu.
    2. Open ID configuration endpoint is available in the Applications, Advanced Settings, Endpoints tab.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

root@find-nuclei-web-server:~# cat /etc/default/minio
MINIO_VOLUMES="/usr/local/share/minio/"
MINIO_OPTS="-C /etc/minio --address :9000 --console-address :9001"
MINIO_ROOT_USER="************"
MINIO_ROOT_PASSWORD="*******"
MINIO_IDENTITY_OPENID_CONFIG_URL=ADD_YOUR_ENDPOINT
MINIO_IDENTITY_OPENID_CLIENT_ID="*********"
MINIO_IDENTITY_OPENID_CLIENT_SECRET="***********"
MINIO_BROWSER_REDIRECT_URL="https://minio.domain-name.com"
MINIO_IDENTITY_OPENID_SCOPES="openid,profile,email,offline_access"
MINIO_IDENTITY_OPENID_CLAIM_PREFIX="https://minio/"
DIGITALOCEAN_ENDPOINT="Check Spaces settings for end point"
  1. Restart MinIO, head to minio.domain-name.com:

    MinIO SSO Login

    MinIO Auth0 Login

    DigitalOcean Spaces in MinIO Console

Debugging

MinIO Client debugging for when things go wrong

Install MinIO client on your local laptop or the MinIO Server depending on which step you have issues.

If you’re debugging MinIO sever startup with no NGINX configuration, etc. use the client on the server:

1

mc alias set local http://localhost:9000 KEY SECRET

If you have NGINX config set, SSL certs provisioned you can setup a client on you local machine:

1

mc alias set minio-server https://minio-server.domain-name.com KEY SECRET

Example use:

1
2
3
4
5
6
7

root@workstation:~# ./mc ls minio-server/vi-object-store
[2021-12-18 15:00:43 CET] 1.0MiB IMG_5746.jpg
[2022-01-01 01:28:56 CET]  30KiB droplet_cost (1).png
[2022-01-01 02:46:31 CET]     0B Test_Folder/

root@workstation:~# ./mc du minio-server/vi-object-store
2.1MiB  vi-object-store

Client usage

  1. List current policies
1
2
3
4
5
6

mc admin policy list local
  readwrite
  writeonly
  consoleAdmin
  diagnostics
  readonly
  1. Trace server. This command will allow you to see the requests and the error codes and messages in case of any issues.
1

mc admin trace -v -a minio-server

You can find more here.

Testing Auth0 with API debugger

Auth0 API debugger extension is an incredibly useful tool especially if you’re trying to add some custom content to the tokens, like MinIO policy claims. Follow the [instructions]((https://auth0.com/docs/extensions/authentication-api-debugger-extension) to learn how to enable and use.

Some example use below:

  1. Launch the extension

    Extension list

  2. Check the config:

    Config

  3. Get the token, in OAuth2 / OIDC:
    1. Set Response Type to “code”
    2. Use “OAUTH2/OIDC LOGIN” User Flow (Blue button top of the page)

      Code

  4. After getting successful response go back to Login then OAuth2 / OIDC
    1. Authorization Code should be filled in from the previous request
    2. Set Response Type to “id_token”
    3. Use “OAUTH2 CODE EXCHANGE” User Flow (Blue button top of the page) to get the token payload:

      ID TOKEN

Updated on 2021-12-12
Read Markdown
 MinIO, Storage, Data Management, Auth0, DigitalOcean
Back | Home