Skip to content

Conversation

stevapple
Copy link

@stevapple stevapple commented Mar 7, 2025

Description

LDAP support exists in Apache-2.0 MinIO, but it's lacking some key functionalities. In this PR we add:

  • support for listing valid users with LDAP;
  • support for STS policies with LDAP;
  • support for logging into the browser with LDAP STS;
  • support for listing groups with LDAP.

This PR is the core of unblocking juicedata/juicefs#5368

Motivation and Context

See juicedata/juicefs#5368

How to test this PR?

Suppose you have an LDAP server running on ldaps://ldap.example.com:636 with root DN dc=example,dc=com, and a test FS sqlite3://test.db.

# Build
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN="ou=groups,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER="(&(objectClass=groupOfUniqueNames)(uniqueMember=%d))"
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN="cn=juicefs,ou=services,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD="jfs-password"
export MINIO_IDENTITY_LDAP_SERVER_ADDR="ldap.example.com:636"
export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN="ou=users,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER="(&(objectClass=inetOrgPerson)(uid=%s))"
export MINIO_ROOT_USER="admin"
export MINIO_ROOT_PASSWORD="juicefs"
juicefs gateway sqlite3://test.db localhost:9000 --multi-buckets

You can use the browser to log into the gateway via http://localhost:9000/minio.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation updated
  • Unit tests added/updated

@stevapple stevapple marked this pull request as ready for review March 7, 2025 05:10
@zhijian-pro zhijian-pro requested a review from eltonxiao124 May 30, 2025 03:44
@Xuyuchao-juice
Copy link

I failed to test successfully using your method. Could you please help me see where the problem lies

Steps to Reproduce

  1. Setup LDAP Server

    docker run -d --name openldap \
      -p 5389:389 \
      -e LDAP_ORGANISATION="JuiceFS Test" \
      -e LDAP_DOMAIN="example.com" \
      -e LDAP_ADMIN_PASSWORD="123" \
      osixia/openldap:1.5.0
    sleep 15  # Wait for service initialization
  2. Configure LDAP Test Data
    Executed the script below to populate users/groups:

    #!/bin/bash
    
    if ! ldapadd -x -H ldap://0.0.0.0:5389 -D "cn=admin,dc=example,dc=com" -w 123 <<EOF
       dn: ou=users,dc=example,dc=com
       objectClass: organizationalUnit
       ou: users
    
       dn: ou=groups,dc=example,dc=com
       objectClass: organizationalUnit
       ou: groups
    
       dn: cn=jfsusers,ou=groups,dc=example,dc=com
       objectClass: groupOfUniqueNames
       cn: jfsusers
       uniqueMember: cn=user1,ou=users,dc=example,dc=com
       uniqueMember: cn=user2,ou=users,dc=example,dc=com
    
       dn: cn=user1,ou=users,dc=example,dc=com
       objectClass: inetOrgPerson
       cn: user1
       sn: user1
       uid: user1
       userPassword: pass1
    
       dn: cn=user2,ou=users,dc=example,dc=com
       objectClass: inetOrgPerson
       cn: user2
       sn: user2
       uid: user2
       userPassword: pass2
       EOF
    then
       echo "error" >&2
       exit 1
    fi
    
    echo "LDAP数据结构添加成功!"

    Verification:
    ✅ Confirmed entries: user1 (pass1), user2 (pass2), group jfsusers.

  3. Build & Configure JuiceFS Gateway

    git checkout <PR_BRANCH>
    make build
    
    export MINIO_IDENTITY_LDAP_SERVER_ADDR="0.0.0.0:5389"
    export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN="ou=users,dc=example,dc=com"
    export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER="(&(objectClass=inetOrgPerson)(uid=%s))"
    export MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN="ou=groups,dc=example,dc=com"
    export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER="(&(objectClass=groupOfUniqueNames)(uniqueMember=%d))"
    export MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN="cn=admin,dc=example,dc=com"  # Fixed missing account
    export MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD="123"
    export MINIO_ROOT_USER="admin"
    export MINIO_ROOT_PASSWORD="juicefs123"
    
    juicefs format sqlite3://test.db myjfs
    ./juicefs gateway sqlite3://test.db 0.0.0.0:9000 --multi-buckets -v  # Added verbose logging
  4. Attempt Login

    • Accessed WebUI: http://localhost:9000/minio
    • Tried credentials:
      • Access Key = user1, Secret Key = pass1
      • Access Key = user2, Secret Key = pass2
        ❌ Result: Invalid credentials error.

@stevapple
Copy link
Author

@Xuyuchao-juice If you're using plain-text LDAP (port 389), you'll need to specify MINIO_IDENTITY_LDAP_SERVER_INSECURE explicitly:

export MINIO_IDENTITY_LDAP_SERVER_INSECURE=true

The default protocol for MinIO is LDAPS. Have you noticed any connection failure in the gateway log?

@Xuyuchao-juice
Copy link

I added MINIO_IDENTITY_LDAP_SERVER_INSECURE=true to the environment variable, but it still didn't work.
gateway startup does have an error, the log is as follow :

root@iZ92wx62d5s1ruZ:~/code/juicefs# ./juicefs gateway sqlite3://test.db 0.0.0.0:9000 --multi-buckets
2025/06/30 22:14:22.852594 juicefs[960381] <INFO>: Meta address: sqlite3://test.db [NewClient@interface.go:578]
2025/06/30 22:14:22.857308 juicefs[960381] <INFO>: Data use file:///var/jfs/myjfs/ [initForSvc@gateway.go:245]
2025/06/30 22:14:22.857850 juicefs[960381] <INFO>: Disk cache (/var/jfsCache/9ce55670-b25a-4fd9-aafb-4d209edfe460/): used ratio - [space 79.6%, inode 15%] [newCacheStore@disk_cache.go:147]
2025/06/30 22:14:22.857928 juicefs[960381] <INFO>: Adjusted cache capacity based on freeratio: from 107374182400 to 37694648140 bytes [setlimitByFreeRatio@disk_cache.go:171]
2025/06/30 22:14:22.857954 juicefs[960381] <INFO>: Adjusted max items based on freeratio: from 0 to 2347329 items [setlimitByFreeRatio@disk_cache.go:176]
2025/06/30 22:14:22.861218 juicefs[960381] <INFO>: Create session 20 OK with version: 1.3.0-dev+2025-06-06.ccd547d7 [NewSession@base.go:535]
2025/06/30 22:14:22.862102 juicefs[960381] <INFO>: Prometheus metrics listening on 127.0.0.1:40327 [exposeMetrics@mount.go:134]

API: SYSTEM(bucket=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5, object=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5)
Time: 22:14:22 CST 06/30/2025
RemoteHost: 5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5
Error: *fmt.wrapError
      16: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config-current.go:373:cmd.lookupConfigs()
      15: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config-current.go:606:cmd.loadConfig()
      14: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config.go:242:cmd.initConfig()
      13: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config.go:196:cmd.(*ConfigSys).Init()
      12: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/server-main.go:335:cmd.initAllSubsystems()
      11: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/server-main.go:297:cmd.initServer()
      10: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/server-main.go:600:cmd.ServerMainForJFS()
       9: /root/code/juicefs/cmd/gateway.go:218:cmd.gateway2()
       8: /root/go/pkg/mod/github.com/minio/cli@v1.24.2/app.go:490:cli.HandleAction()
       7: /root/go/pkg/mod/github.com/minio/cli@v1.24.2/app.go:269:cli.(*App).Run()
       6: /root/code/juicefs/cmd/gateway.go:212:cmd.gateway()
       5: /root/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/command.go:177:v2.(*Command).Run()
       4: /root/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:389:v2.(*App).RunContext()
       3: /root/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:254:v2.(*App).Run()
       2: /root/code/juicefs/cmd/main.go:101:cmd.Main()
       1: /root/code/juicefs/main.go:29:main.main()
Endpoint: http://0.0.0.0:9000 

Browser Access:
   http://0.0.0.0:9000

Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
IAM initialization complete

@929496959
Copy link

Currently, when using Windows PowerShell and executing the command "minio.exe server ./data", an error is reported
`PS C:\Users\Administrator\Desktop\minio-new-base\miniobin> .\minio.exe server ./data

+---------------------------------------------------------------+
| You are running an older version of MinIO released 6 days ago |
| Update: Run mc admin update |
+---------------------------------------------------------------+

Waiting for all MinIO sub-systems to be initialized.. possible cause (Unable to list buckets to heal: disk not found)
Waiting for all MinIO sub-systems to be initialized.. trying to acquire lock
Waiting for all MinIO sub-systems to be initialized.. trying to acquire lock
Exiting on signal: INTERRUPT

API: SYSTEM()
Time: 10:46:41 CST 08/13/2025
DeploymentID: 6f16890a-c61e-4ca2-9bfe-ae8e15bbb42a
Error: Operation timed out (cmd.OperationTimedOut)
1: C:\Users\Administrator\Desktop\minio-new-base\cmd\data-scanner.go:144:cmd.runDataScanner()`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants