Input optional subcommands without order

[vwxyzjn]

Hi @brentyi,

Is it possible to do the following in 03_multiple_subcommands.py

python 03_multiple_subcommands.py optimizer:adam --optimizer.learning-rate 0.1

We have cases where we'd like to use the default optimizer:adam as is.

[brentyi]

Hi Costa! Unfortunately no, to specify the later subcommands we currently need the earlier ones.

I think supporting this is doable, but the implementation details would require some thought.

Depending on application it can also be hacked, for example:

if __name__ == "__main__":
    import sys

    args = sys.argv[1:]
    if "dataset:" not in args[0]:
        args = ["dataset:mnist"] + args
    tyro.cli(train, config=(tyro.conf.ConsolidateSubcommandArgs,), args=args)

[vwxyzjn]

Thanks for the reply. Supporting it would be more intuitive to use, since we'd usually like to use the default. No worries if you couldn't get to it, tyro is awesome as it is :)

[tmke8]

I also wish for this feature, in particular in the context of having multiple base configs. Consider this variation on 03_multiple_subcommands.py:

import dataclasses
from typing import Literal
import tyro

@dataclasses.dataclass
class Mnist:
    binary: bool = False

@dataclasses.dataclass
class ImageNet:
    subset: Literal[50, 100, 1000]

@dataclasses.dataclass
class Adam:
    lr: float = 1e-3
    betas: tuple[float, float] = (0.9, 0.999)

@dataclasses.dataclass
class Sgd:
    lr: float = 3e-4

@dataclasses.dataclass
class Experiment:
    dataset: Mnist | ImageNet = dataclasses.field(default_factory=Mnist)
    optimizer: Adam | Sgd = dataclasses.field(default_factory=Sgd)

    def train(self) -> None:
        print(dataclasses.asdict(self))

default_configs = {
    "mnist-base": Experiment(dataset=Mnist(binary=True), optimizer=Adam(lr=3e-4)),
    "image-net-base": Experiment(dataset=ImageNet(subset=100), optimizer=Adam(lr=1e-4)),
}

if __name__ == "__main__":
    exp = tyro.cli(
        tyro.extras.subcommand_type_from_defaults(default_configs),
        config=(tyro.conf.ConsolidateSubcommandArgs,))
    exp.train()

I would then like to run

python 03_multiple_subcommands.py mnist-base --optimizer.lr 1e-2

where I want to let the first subcommand, mnist-base, choose the appropriate values of the subsequent subcommands, so to speak. But this doesn't work and instead I have to use

python 03_multiple_subcommands.py mnist-base dataset:mnist optimizer:adam --optimizer.lr 1e-2

which is very verbose and repeats information that is already in the base config.

The simple hack mentioned above doesn't work here because the subcommands that we'd have to add to args depend on the first command. I suppose I could do something like this:

    args = sys.argv[1:]
    if "dataset:" not in args[1]:
        if args[0] == "mnist-base":
            args.insert(1, "dataset:mnist")
        elif args[0] == "image-net-base":
            args.insert(1, "dataset:imagenet")
    if "optimizer" not in args[2]:
       # ...

but this is getting quite complicated and brittle.

[brentyi]

Hi @tmke8!

Can you see if tyro.extras.overridable_config_cli() helps? https://brentyi.github.io/tyro/api/tyro/extras/#tyro.extras.overridable_config_cli

You can also see how it's implemented here, the important bit is the tyro.conf.AvoidSubcommands flag:

tyro/src/tyro/extras/_base_configs.py

Lines 90 to 105 in 7a6c0bf

	return tyro.cli(
	tyro.extras.subcommand_type_from_defaults(
	defaults={k: configs[k][1] for k in keys},
	descriptions={k: configs[k][0] for k in keys},
	sort_subcommands=sort_subcommands,
	),
	prog=prog,
	description=description,
	args=args,
	use_underscores=use_underscores,
	console_outputs=console_outputs,
	# Don't create subcommands for union types within the config object.
	config=(tyro.conf.AvoidSubcommands,)
	+ (tuple() if config is None else tuple(config)),
	registry=registry,
	)

The downside here is you won't be able to switch between the union types within your base config anymore.

[tmke8]

Yes, as you say, this helps in the sense that I won't have to specify dataset:... anymore, but I lose the ability to select a different optimizer. I would then need to create base configs for all combinations of unions that I want to try (i.e., mnist-adam, mnist-sgd, imagenet-adam, imagenet-sgd).