How We Optimized ACL Implementation for Minimal Performance Impact

In JuiceFS 1.2 Beta 1, we introduced support for POSIX access control lists (ACLs). You can use this feature to achieve more fine-grained permission control.

From early UNIX-like systems to the present day, standard permissions (OWNER, GROUP, OTHERS) are most commonly used; special permissions (SUID, SGID, STICKY) are used for more complicated scenarios. With the introduction of ACLs, you can combine policies more freely to achieve authorizations that standard permissions alone cannot accomplish. In this post, we’ll deep dive into ACL structures, implementation details, and performance benchmarks. You’ll learn how we optimized ACL implementation for minimal performance impact.

About ACLs

ACLs follow POSIX 1003.2c draft 17, with slight variations across different systems. We use the Linux system’s implementation as the standard.

ACLs have two categories:

• Access ACLs: Used for defining and checking permissions, as described below and in the default ACL semantics.
• Default ACLs: Structurally similar to access ACLs but applicable only to directories. Access ACLs for child files/directories inherit from their parent directory’s default ACLs.

The table below shows the ACL structure. Each configuration (each row) is an entry.

ClassEntry typeText formowner classOwneruser::rwxgroup classNamed useruser:name:rwxgroup classOwning groupgroup::rwxgroup classNamed groupgroup:name:rwxgroup classMaskmask::rwxother classOthersother::rwx

Compared to standard permission mode, ACLs adds three entries, marked in bold:

• The named user entry: It allows specifying permissions for multiple users, extending beyond the permissions limited to the owner.
• The named group: Similar to the named user entry, it extends permissions to user groups.
• The mask entry: It is quite special and will be described in detail later. It’s different from the umask, when ACLs are set, the umask is ignored.

ACLs and permission mode are fully compatible. We can categorize them as follows based on definitions:

• Minimal/minimum ACLs: Equivalent to the standard permission mode.
• Extended ACLs: Configured with any named user or named group entries.

The following operations show the relationship between ACLs and permission mode.

# Check permission mode.
$ ls -l
drwxr-xr-x 2 root root 4.0K  April 12 09:32 d1

# Use setfacl to set group1 permissions for the d1 directory.
$ setfacl -m g:group1:rwx d1
# Check permission mode gain.
$ ls -l
drwxrwxr-x+ 2 root root 4.0K  April 12 09:32 d1

After setting ACLs, the group class permissions in the file permission mode have changed, adding the “w” permission. This change occurs because ACLs extend the group class (see the preceding table).

The group class in the permission mode only represents the permissions of the owner group. But in ACLs, it represents the upper limit of permissions for all entries in the group class. Therefore, the permissions in the group class change with changes in ACL entries.

As mentioned earlier, ACLs introduce a new entry named mask, which represents the dynamic permission changes in the group class. When ACLs are set, the permissions displayed in the group class of the permission mode correspond to the mask entry.

By default, when setting ACLs using setfacl, the mask is automatically calculated. For example, as shown earlier, mask = owner group entry | group1 entry, which results in the union “rwx”. Alternatively, the mask can be set independently. In such cases, the final permissions for entries in the group class need to intersect with the mask:

• The final effective permissions for the owner group: “r-x” & “-wx” = “ — x”
• The permissions for group1: “rwx” & “-wx” = “-wx”

# Following the previous operations, set the mask to "-wx."
$ setfacl -m m::-wx d1

# Check the ACL.
$ getfacl d1 --omit-header
user::rwx
group::r-x                      #effective:--x
group:group1:rwx                #effective:-wx
mask::-wx
other::r-x

Implementation of ACLs

ACLs provide finer-grained permission management but also reduce the performance. Therefore, this section will dive into the optimization strategies implemented by JuiceFS in terms of storage space and performance.

ACL reuse

ACLs are directly associated with files and can be viewed as an extended file attribute. In some file system implementations, each file maintains a separate copy of ACLs in extended attributes (xattr). This increases a lot of metadata.

JuiceFS defines the storage structure for ACLs as follows. If there are tens of thousands of files in the file system, and each file saves a configuration, storing a corresponding ACL for each file becomes excessively bulky and inefficient.

type ACL struct {
    Owner       uint16
    Group       uint16
    Mask        uint16
    Other       uint16
    NamedUsers  []Entry
    NamedGroups []Entry
}

type Entry struct {
    Id   uint32 // User ID or group ID
    Perm uint16
}

In common application scenarios, ACL configurations are not too many. In a directory using the default ACL, file objects created under it all inherit the same access ACL.

Therefore, JuiceFS has implemented ACL reuse optimization. Specifically, we decouple ACLs from xattr, store them in a separate metadata engine structure, and map the same ACLs to a globally unique configuration. This significantly reduces the storage requirements for duplicate data. Within a file object, only an ACL ID is stored. This ACL ID is assigned by the metadata engine to ensure global uniqueness and increment. It effectively avoids conflicts between multiple clients (such as multiple mount points). Combining this with local caching, the performance degradation of ACL permission checks can be reduced to a level comparable to standard permissions. In addition, this approach enhances the performance of creating file objects in default ACL scenarios, since only one ACL ID needs to be assigned for file objects. This reduces the frequency of ACL storage requests handled by the metadata engine.

ACL cache

After enabling ACLs, every file object operation requires the kernel to obtain the ACL and then perform ACL permission checks. Therefore, efficient ACL retrieval is crucial. Retrieving ACL configurations from the metadata engine for each request would decrease performance.

Based on the ACL reuse premise, JuiceFS caches all ACL configurations on the client side. Furthermore, file object attributes store an ACL ID. Reading the ACL requires only a simple in-memory indexing operation. In the current implementation, ACLs are only added, not modified. Therefore, ACL cache among different clients only requires synchronization of newly added ACL configurations. If the cache fails to read data, missing ACL configurations are loaded from the metadata engine. Because adding ACL configurations is an infrequent operation, the performance decrease in this area is controllable.

Minimal ACLs

In a file system with tens of thousands of file objects, stricter ACL management may only be required for a very small number of files. Other files can effectively operate with minimal ACLs, which aligns their management with the original permission mode.

Therefore, in the JuiceFS implementation, we’ve logically ensured compatibility. For minimal ACLs, actual creation and storage of ACLs are not performed, keeping the performance similar to when the ACL functionality is not enabled.

Performance testing

Enabling ACLs inevitably adds some performance burden. Therefore, we conducted performance testing to compare before and after enabling ACL functionality. The results show that the performance degradation was about 5%. The testing configuration is as follows:

• Machine configuration: 4 cores, 16 GB RAM
• Metadata engine: Redis
• Dataset: 1 million files, 100 files per folder
• ACL configurations: 400 users and default users, 200 groups and default groups

The table below shows the testing results:

OperationACL (op/s)No ACL (op/s)create7,4367,965mkdir6,9767,451open1,2191,173rename409424ls1,4421,490delete968919chmod1,2191,220hardlink740765symlink8,9728,877setxattr1,2081,298

ACL usage

To enable the ACL functionality, you can refer to the POSIX ACL document, and then use the setfacl and getfacl tools to set and view ACL configurations.

ACL permission checks are similar to permission checks, matching each entry individually. For rule details, see Access Check Algorithm.

Give it a try!

Feel free to download and try it out. If you have any feedback or suggestions, you can join JuiceFS discussions on GitHub and our community on Slack.