:multiple does not work with :default

[ieugen]

Problem

I tried to use a multiple with default value and I get an error.

Also the error message is not very helpful: No value supplied for key: cli_matic.utils$assoc_new_multivalue@a520a55

Repro

(def demographics-cli-matic-config
  {:command "demographics"   
   :subcommands [{:command "run"
                  :opts [{:as      "The demograpics type."
                          ;; :default ["ethnicity" "otherp"]
                          :option  "type"
                          :multiple true
                          :type    :string}]
                  :runs println}]})

{:clojure.main/message
 "Execution error (IllegalArgumentException) at clojure.tools.cli/compile-spec (cli.cljc:259).\nNo value supplied for key: cli_matic.utils$assoc_new_multivalue@a520a55\n",
 :clojure.main/triage
 {:clojure.error/class java.lang.IllegalArgumentException,
  :clojure.error/line 259,
  :clojure.error/cause
  "No value supplied for key: cli_matic.utils$assoc_new_multivalue@a520a55",
  :clojure.error/symbol clojure.tools.cli/compile-spec,
  :clojure.error/source "cli.cljc",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type java.lang.IllegalArgumentException,
    :message
    "No value supplied for key: cli_matic.utils$assoc_new_multivalue@a520a55",
    :at
    [clojure.lang.PersistentHashMap
     create
     "PersistentHashMap.java"
     77]}],
  :trace
  [[clojure.lang.PersistentHashMap create "PersistentHashMap.java" 77]
   [clojure.core$hash_map invokeStatic "core.clj" 389]
   [clojure.core$hash_map doInvoke "core.clj" 381]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$apply invoke "core.clj" 662]
   [clojure.tools.cli$compile_spec invokeStatic "cli.cljc" 259]
   [clojure.tools.cli$compile_spec invoke "cli.cljc" 257]
   [clojure.tools.cli$compile_option_specs$fn__17275
    invoke
    "cli.cljc"
    345]
   [clojure.core$map$fn__5935 invoke "core.clj" 2770]
   [clojure.lang.LazySeq sval "LazySeq.java" 42]
   [clojure.lang.LazySeq seq "LazySeq.java" 51]
   [clojure.lang.RT seq "RT.java" 535]
   [clojure.core$seq__5467 invokeStatic "core.clj" 139]
   [clojure.core$every_QMARK_ invokeStatic "core.clj" 2696]
   [clojure.core$every_QMARK_ invoke "core.clj" 2689]
   [clojure.tools.cli$compile_option_specs invokeStatic "cli.cljc" 335]
   [clojure.tools.cli$compile_option_specs invoke "cli.cljc" 291]
   [clojure.tools.cli$parse_opts invokeStatic "cli.cljc" 753]
   [clojure.tools.cli$parse_opts doInvoke "cli.cljc" 564]
   [clojure.lang.RestFn invoke "RestFn.java" 464]
   [cli_matic.core$parse_cmds_with_defaults
    invokeStatic
    "core.cljc"
    145]
   [cli_matic.core$parse_cmds_with_defaults invoke "core.cljc" 110]
   [cli_matic.core$parse_cmds_with_positions
    invokeStatic
    "core.cljc"
    169]
   [cli_matic.core$parse_cmds_with_positions invoke "core.cljc" 155]
   [cli_matic.core$parse_command_line invokeStatic "core.cljc" 320]
   [cli_matic.core$parse_command_line invoke "core.cljc" 266]
   [cli_matic.core$run_cmd_STAR_ invokeStatic "core.cljc" 575]
   [cli_matic.core$run_cmd_STAR_ invoke "core.cljc" 560]
   [cli_matic.core$run_cmd invokeStatic "core.cljc" 601]
   [cli_matic.core$run_cmd invoke "core.cljc" 591]
   [dre.app_tasks.main$_main invokeStatic "main.clj" 59]
   [dre.app_tasks.main$_main doInvoke "main.clj" 56]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.main$main_opt invokeStatic "main.clj" 514]
   [clojure.main$main_opt invoke "main.clj" 510]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause
  "No value supplied for key: cli_matic.utils$assoc_new_multivalue@a520a55"}}

Execution error (IllegalArgumentException) at clojure.tools.cli/compile-spec (cli.cljc:259).
No value supplied for key: cli_matic.utils$assoc_new_multivalue@a520a55

Expected vs actual behavior

demographics run --type aa --type bb
{:type [aa bb], :_arguments []}

Version / Platform

java -version
openjdk version "11.0.12" 2021-07-20
OpenJDK Runtime Environment 18.9 (build 11.0.12+7)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.12+7, mixed mode)

[ieugen]

I added this to test/cli_matic/presets_test.cljc

(comment

  (test-string)

  (use 'clojure.tools.trace)
  (trace-ns cli-matic.core)
  (trace-ns cli-matic.utils)
  (trace-ns cli-matic.utils-v2)
  (untrace-ns clojure.tools.cli)

  (parse-cmds-simpler
   ["foo"]
   (mkDummyCfg {:option "val" :as "x" :type :string :multiple true
                :default ["a" "b" "c"]}))
  
    (parse-cmds-simpler
     ["foo" "--val" "x" "--val" "y"]
     (mkDummyCfg {:option "val" :as "x" :type :string :multiple true
                  :default ["a" "b" "c"]}))
  )

And it seems the issue might be related to mk-cli-option in src/cli_matic/utils.cljc .

TRACE t10880: | | (cli-matic.core/parse-cmds-with-defaults [{:option "val", :as "x", :type :string, :multiple true, :default ["a" "b" "c"]}] [] false #function[cli-matic.platform/read-env])
TRACE t10881: | | | (cli-matic.utils/cm-opts->cli-opts [{:option "val", :as "x", :type :string, :multiple true, :default ["a" "b" "c"]}])
TRACE t10882: | | | | (cli-matic.utils/mk-cli-option {:option "val", :as "x", :type :string, :multiple true, :default ["a" "b" "c"]})
TRACE t10883: | | | | | (cli-matic.utils/asString "x")
TRACE t10883: | | | | | => "x"
TRACE t10884: | | | | | (cli-matic.utils/get-cli-option :string)
TRACE t10884: | | | | | => {:placeholder "S"}
TRACE t10885: | | | | | (cli-matic.utils/mk-short-opt nil)
TRACE t10885: | | | | | => nil
TRACE t10886: | | | | | (cli-matic.utils/mk-long-opt "val" "S" :string)
TRACE t10886: | | | | | => "--val S"
TRACE t10887: | | | | | (cli-matic.utils/mk-env-name "x" nil false)
TRACE t10887: | | | | | => "x"
TRACE t10882: | | | | => [nil "--val S" "x" :default "a" "b" "c" :assoc-fn #function[clojure.tools.trace/trace-var*/fn--8942/tracing-wrapper--8943]]
TRACE t10881: | | | => [[nil "--val S" "x" :default "a" "b" "c" :assoc-fn #function[clojure.tools.trace/trace-var*/fn--8942/tracing-wrapper--8943]] ["-?" "--help" "" :id :_help_trigger]]
TRACE t10880: | | => {:options {:val "a"}, :arguments [], :summary "      --val S  a  x\n  -?, --help", :errors nil}

[ieugen]

The issue is with the use of flatten call in the case of multiple .

(defn mk-cli-option
  "Builds a tools.cli option out of our own format.

  If for-parsing is true, the option will be used for parsing;
  if false, for generating help messages.

  "
  [{:keys [option short as type default multiple env]}]

  (let [as_description (asString as)
        preset (get-cli-option type)
        placeholder (str (:placeholder preset)
                         (if (= :present default) "*" ""))
        positional-opts [(mk-short-opt short)
                         (mk-long-opt option placeholder type)
                         (mk-env-name as_description env false)]

        ;; step 1 - remove :placeholder
        opts-1 (dissoc preset :placeholder)

        ;; step 2 - add default if present and is not ":present"
        opts-2 (if (and (some? default)
                        (not= :present default))
                 (assoc opts-1 :default default)
                 opts-1)
        ;; step 3 - if multivalue, add correct assoc-fns
        opts-3 (if multiple
                 (assoc opts-2 :assoc-fn assoc-new-multivalue)
                 opts-2)]
    (println "p-opts" positional-opts)
    (println "1" opts-1)
    (println "2" opts-2)
    (println "3" opts-3)
    (apply
     conj positional-opts
     (flatten (seq opts-3)))))

(comment

  (use 'clojure.tools.cli)

  (let [opt
        (mk-cli-option
         {:option "val", :as "x", :type :string, :multiple true, :default ["a" "b" "c"]})]
    (println "optsss" opt)
    (parse-opts [] [opt]))
  
  )

Results;

p-opts [nil --val S x]
1 {}
2 {:default [a b c]}
3 {:default [a b c], :assoc-fn #function[cli-matic.utils/assoc-new-multivalue]}
optsss [nil --val S x :default a b c :assoc-fn #function[cli-matic.utils/assoc-new-multivalue]]
{:options {:val "a"}, :arguments [], :summary "      --val S  a  x", :errors nil}
; Warning: The following options to parse-opts are unrecognized: b

[ieugen]

The issue is with flatten.

[ieugen]

Seems like the issue is with tools-cli or my understanding of how to use it.

Copied the example from the repo and added a value to :default for -f .

(def cli-options
  [
   ["-f" "--file NAME" "File names to read"
    :multi true ; use :update-fn to combine multiple instance of -f/--file
    :default ["test"]
    ;; with :multi true, the :update-fn is passed both the existing parsed
    ;; value(s) and the new parsed value from each option
    :update-fn conj]
   ])

(parse-opts args cli-options)

Parsing cli args using the above structure yields both default and the user supplied options.

clj -M -m cli.example -f csv

{:options {:file [test csv]}, :arguments [], :summary  
  -f, --file NAME      ["test"]   File names to read
  :errors nil}