chore(docs): FilePermissions cas config user documentation (#128)

* This commit does the following changes
 - Add FilePermissions cas config user documentation
 - Add FilePermission key sample in StorageClass
 - Update non-root usage instructions

Signed-off-by: Niladri Halder <niladri.halder@mayadata.io>

Author: niladrih

deploy/kubectl/busybox-openebs-rwx.yaml

@@ -5,8 +5,10 @@ metadata:
   name: openebs-rwx-pvc
   #annotaions:
   #  cas.openebs.io/config: |
-  #      FilePermissions can be used to modify the owner, group
-  #      and file modes of the shared NFS filesystem.   
+  #     FilePermissions defines the file ownership and mode specifications
+  #     for the NFS server's shared filesystem volume.
+  #     File permission changes are applied recursively if the root of the
+  #     volume's filesystem does not match the specified value.
   #    - name: FilePermissions
   #      data:
   #        UID: "1000"

deploy/kubectl/openebs-nfs-provisioner.yaml

@@ -202,27 +202,39 @@ metadata:
         value: "kernel"
       - name: BackendStorageClass
         value: "openebs-hostpath"
-      #  NFSServerResourceRequests defines the resource requests for NFS Server
+      # NFSServerResourceRequests defines the resource requests for NFS Server
       #- name: NFSServerResourceRequests
       #  value: |-
       #      memory: 50Mi
       #      cpu: 50m
-      #  NFSServerResourceLimits defines the resource limits for NFS Server
+      # NFSServerResourceLimits defines the resource limits for NFS Server
       #- name: NFSServerResourceLimits
       #  value: |-
       #      memory: 100Mi
       #      cpu: 100m
-      #  LeaseTime defines the renewl period(in seconds) for client state
+      # LeaseTime defines the renewal period(in seconds) for client state
       #- name: LeaseTime
       #  value: 30
-      #  GraceTime defines the recovery period(in seconds) to reclaim locks
+      # GraceTime defines the recovery period(in seconds) to reclaim locks
       #- name: GraceTime
       #  value: 30
-      #  FSGID defines the group permissions of NFS Volume. If it is set
-      #  then non-root applications should add FSGID value under pod
-      #  Suplemental groups.
-      #  The FSGID config key is being deprecated. Please use the
-      #  FilePermissions config key on the PersistentVolumeClaim instead.
+      # FilePermissions defines the file ownership and mode specifications
+      # for the NFS server's shared filesystem volume.
+      # File permission changes are applied recursively if the root of the
+      # volume's filesystem does not match the specified value.
+      # Volume-specific file permission configuration can be specified by
+      # using the FilePermissions config key in the PVC YAML, instead of
+      # the StorageClass's.
+      #- name: FilePermissions
+      #  data:
+      #    UID: "1000"
+      #    GID: "2000"
+      #    mode: "0744"
+      # FSGID defines the group permissions of NFS Volume. If it is set
+      # then non-root applications should add FSGID value under pod
+      # Suplemental groups.
+      # The FSGID config key is being deprecated. Please use the
+      # FilePermissions config key instead.
       #- name: FSGID
       #  value: "120"
 provisioner: openebs.io/nfsrwx

docs/troubleshooting/non-root-application-accesing-nfs-volume.md

@@ -13,44 +13,31 @@ To support above use cases OpenEBS NFS Provisioner provides an option to configu
 
 Non-root applications can consume NFS volume by following two steps:
 
-- Create a StorageClass by specifying appropriate permissions under FSGID.
-  Volumes provisioned by using below storage class will have permissions set to 120.
-```yaml
-#Sample storage classes with OpenEBS LVM-LocalPV
-apiVersion: storage.k8s.io/v1
-kind: StorageClass
-metadata:
-  name: openebs-non-root-rwx
-  annotations:
-    openebs.io/cas-type: nfsrwx
-    cas.openebs.io/config: |
-      - name: NFSServerType
-        value: "kernel"
-      - name: BackendStorageClass
-        value: "openebs-lvm-localpv"
-      - name: FSGID
-        value: "120"
-provisioner: openebs.io/nfsrwx
-reclaimPolicy: Delete
-```
+- Create a PersistentVolumeClaim by specifying appropriate permissions under the FilePermissions config key.
+  The volumes provisioned for this PVC will have its group-owner set to 120, and its SetGID bit enabled ([read more](../tutorial/file-permissions.md)).
 
-- Create a persistent volume claim with storage class referring to above example
+Sample PersistentVolumeClaim:  
 ```yaml
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
   name: nfs-pvc
+  annotations:
+    cas.openebs.io/config: |
+      - name: FilePermissions
+        data:
+          GID: "120"
+          mode: "g+s"
 spec:
-  storageClassName: openebs-non-root-rwx
+  storageClassName: openebs-kernel-nfs
   accessModes:
     - ReadWriteMany
   resources:
     requests:
       storage: 5G
 ```
 
-
-- Now create an application by specifying `FSGID` value(i.e 120) under supplemental groups.
+- Now create an application by specifying the group ID value(i.e 120) under supplemental groups.
   When supplemental groups are specified corresponding user will be part of the same group
   and it makes volume accessible.
 ```yaml
@@ -81,9 +68,6 @@ spec:
       claimName: nfs-pvc
 ```
 
-**Note**: Above NFS volume is deployed on OpenEBS LVM-Local PV.
-
-
 
 ### How to debug permission denied error?
 
@@ -104,5 +88,3 @@ Following are the steps to find permissions configured on backend volume:
   Modify: 2021-07-09 11:23:41.933968717 +0000
   Change: 2021-07-09 11:23:41.933968717 +0000
   ```
-
-- If backend storage is an offering from CSI, check whether [fsGroupPolicy](https://kubernetes-csi.github.io/docs/support-fsgroup.html#csi-driver-fsgroup-support) is populated with an appropriate value.

docs/tutorial/file-permissions.md

@@ -0,0 +1,63 @@
+# How to set File Permissions for the shared filesystem
+
+You can use the 'FilePermissions' key in the `cas.openebs.io/config` PersistentVolumeClaim annotation to modify the owner, group and file modes of the shared NFS filesystem.
+
+The file permission changes are handled before the NFS server initializes. 
+
+The `chown` and `chmod` commands are run with the `--recursive` flag.<br>
+**NOTE:** The commands are run only if the existing values for the owner, the group or the file mode of the root of the shared directory do not match with the requested values. This is similar to the Kubernetes [fsGroupChangePolicy's "OnRootMismatch"](https://kubernetes.io/blog/2020/12/14/kubernetes-release-1.20-fsgroupchangepolicy-fsgrouppolicy/#allow-users-to-skip-recursive-permission-changes-on-mount).
+
+Declare ownership and file mode change specfications using the UID, GID and mode keys (sample PersistentVolumeClaim below):<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**UID:** This is the owner's user ID. Only valid UIDs are usable.<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**GID:** This is the group ID of the owning group. Only valid GIDs are usable.<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**mode:** Use this to specify the filesystem permission modes. Both octal and alphabet inputs are accepted. E.g. "0755", "g+rw".<br><br>
+All of the keys are optional.
+```yaml
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: secure-vol
+  annotations:   
+    cas.openebs.io/config: |
+      - name: FilePermissions
+        data:
+          UID: "1000"
+          GID: "2000"
+          mode: "0744"
+spec:
+  storageClassName: openebs-kernel-nfs
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: 10Gi
+```
+>**Note:** The FilePermissions config may be used with the `cas.openebs.io/config` StorageClass annotation key as well. If the config option is present on both the StorageClass and the PersistentVolumeClaim, the PersistentVolumeClaim config takes precedence.
+
+## FSGID/fsGroup
+
+The 'FSGID' config key (StorageClass annotations) is **being deprecated and will be removed in future releases**. At present, the FSGID config can be used, but it cannot be used along with FilePermission data keys 'GID' and/or 'mode'.
+
+You can use the FilePermissions data values to change the group ownership and set a SetGID bit. This will result in changes similar to setting the fsGroup key in the NFS server Pod's securityContext (same as the previously-used FSGID `cas.openebs.io/config` key).
+
+The following FilePermissions data values will result in a similar effect in volume permissions as setting fsGroup as '2000' with fsGroupChangePolicy as 'OnRootMismatch'.
+
+```yaml
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: secure-vol
+  annotations:
+    cas.openebs.io/config: |
+      - name: FilePermissions
+        data:
+          GID: "2000"
+          mode: "g+s"
+spec:
+  storageClassName: openebs-kernel-nfs
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: 10Gi
+```