[RFE] Implement printer profiles

[zdohnal]

This issue is a placeholder for implementing printer profiles, which are needed for the case:

• CUPS temporary queues work only with LAN located devices, because the tech uses mDNS for device discovery. However, this situation isn't usually a case for enterprises, where their IT engineers tend to put printers into a separate VLAN - they do it because of security and monitoring. Without any migration plan, these use cases would be left behind if permanent queues are gone and CUPS temporary queues become the only technology in CUPS.

The idea of printer profiles was introduced by Mike Sweet during email communication within OpenPrinting group and distro maintainers and presented during PWG F2F meetups.

Set of features provided by printer profiles:

• access to queues on remote print servers, which are not in local network
• access to printers from different network
• setting default options for a printer
• setting a default printer for specific network
• every setting can be related to a specific network (f.e. show printer P1 if I'm in network A, but show printer P2 if I'm in network B, use a different default printers across the networks etc.)

Tasks:

• implement showing queues defined by printer profiles in cupsEnumDests() CUPS library function (idea: cupsEnumDests() will check /etc/cups/profiles/ and if there are some, add them to the output - this will show profile queues in print dialogs, which use cupsEnumDests())
• implement sharing profile-defined queues in cupsd/sharing cups module in modular design
• implement tests for printer profiles in CUPS test suite
• add printer profiles into CUPS documentation, man pages etc

I will try to implement this feature in meantime, but any other help is welcomed.

@michaelrsweet WDYT? Did I forget about something in tasks or do you have any recommendation what to add?

[michaelrsweet]

@zdohnal We don't want an operation for this, we want it to be in file(s) that can be installed.

[zdohnal]

Hi @michaelrsweet ,
hmm, so the expectation is the clients (lpadmin/cups web ui/control-centers/cups-browsed) will write profile files by themselves and libcups functions only pick them up?
Or should admins write profiles by hand and then save them into /etc/cups/profiles or ~/.cups/profiles/ and possibly distribute the profiles through the organization (via special packages)?

Either way, I can't help to not to see a classic PPD déjavu - in both cases above we need to count on profiles being generated and maintained correctly instead of having a full control over them. We would have to keep stable a whole profile file structure or coordinate with known profile implementations if we make some changes - it looks like a bigger challenge than keeping IPP request format stable.

Ad clients creating profiles - I guess we can add the functionality into lpadmin/cups web ui and cups-browsed, but I'm not sure how GTK or other GUI apps will be interested in writing profiles.

What do you think?

[michaelrsweet]

@zdohnal So the point of my "profiles" featurette in my CUPS slides was not to provide a way to further inject PPDs into the system but to provide a simple way for enterprises to configure/deploy IPP Everywhere queues (pointed at real printers, a central CUPS server, etc.) when mDNS/DNS-SD wasn't feasible - basically duplicating what Apple has done for iOS/iPadOS/macOS and what hundreds of users have been asking for over the years on Linux (re: supporting multiple ServerName directives in client.conf).

We can keep the format dirt simple, e.g.:

# comment
ServerName hostname-or-ip[:port]
Printer {ipp,ipps}://hostname-or-ip[:port]/path

Then libcups just contacts each of the hosts during enumeration and lists any it can contact. The auto-assigned names can use either the reported DNS-SD name (from the printer-dns-sd-name attribute) or the remote printer name (from the printer-name attribute) plus hostname, same as what we do for DNS-SD.

(the same approach would be used for LDAP support, whenever we decide to implement that again using the IETF-published schema...)

[zdohnal]

@zdohnal So the point of my "profiles" featurette in my CUPS slides was not to provide a way to further inject PPDs into the system

That wasn't what I meant by PPD dejavu remark, I'm sorry, if it sounded like that - I'll explain later.

but to provide a simple way for enterprises to configure/deploy IPP Everywhere queues (pointed at real printers, a central CUPS server, etc.) when mDNS/DNS-SD wasn't feasible - basically duplicating what Apple has done for iOS/iPadOS/macOS and what hundreds of users have been asking for over the years on Linux (re: supporting multiple ServerName directives in client.conf).

That would be great if cupsd was able to support this feature - I guess it is exactly the first point from my initial comment - can't we have libcups reading client.conf for multiple 'ServerName's and then provides queues from destinations?

However this feature will set the same default options for such queues in user's print dialog as the default options, which the queue has on the server/device - which sometimes differs from the default options user wants - it will become annoying to set the default options every time you want to print.
So IMO cupsd should have a way how to override the attributes which come from server, to give user more flexibility about such queues - and attributes (or options - I guess it will be more readable for non-IPP people) in profiles can do the trick.

Based on the info above, we still would have needed a new file and a new format (or adding it to printers.conf), f.e. like:

<Printer <name>>
URI <value>
(this one would be applied only if we put them into printers.conf) Printer-is-profile <yes/no>
<new default option1/attribute1> <value1>
.
.
.
<new default optionN/attributeN> <valueN>
</Printer>

Additionally - IIUC the permanent queues are planned to go away in the future and that's the source of other mentioned use cases (different default options, 1 device to N queues), which printer profiles can support in addition to the idea what you meant in CUPS slides (althought IIRC you proposed using printer profiles for other mentioned user cases as well when we were discussing CUPS with other maintainers at the time of forking CUPS).

And if user can change options to set new default values in print dialog, something will need to do it for him (I cannot expect an user creating a xml file by themselves), so the options are following:

1. clients will create profiles by themselves and libcups will read them - we won't need an operation for it, but we will need to count on application to create a valid profile and we will need to keep compatibility for whole profile file, otherwise users will have breakages - this is what I meant by PPD dejavu - developing profiles will be blocked/obstructed by client implementations, which aren't on track with CUPS development and compatibility scope is larger than with IPP request.
2. define IPP request/DBUS method, which clients will use to create a profile, and cupsd will create a profile as it sees fit - the pros will be an united profile generation (so not every client needs to implement it) under CUPS. The problem will be to adjust clients to use it (the same is for the former option) and to keep compatibility in IPP request, but that's smaller scope than a file structure.

IMHO it is more feasible to adjust clients to do a request than implement a profile writer, so that's why I proposed a new request type - but I wasn't that sure if we aren't restricted by RFC and PWG standards for IPP, but I didn't find IPP_OP_CREATE_LOCAL_PRINTER operation in the standard either, so I guess it's fine.

Then libcups just contacts each of the hosts during enumeration and lists any it can contact. The auto-assigned names can use either the reported DNS-SD name (from the printer-dns-sd-name attribute) or the remote printer name (from the printer-name attribute) plus hostname, same as what we do for DNS-SD.

IMO if we can decide what to use based on path in uri. If it contains /printers/, use remote printer name, otherwise use DNS-SD name if available.

(the same approach would be used for LDAP support, whenever we decide to implement that again using the IETF-published schema...)

I'm not aware about anyone who uses LDAP for printing right now, so we can leave it out for now - we can reevaluate if there is a demand.

@michaelrsweet Thoughts?

[jsmeix]

@zdohnal
thank you so much for your explanatory description of the issue.
It helps so much to better understand possible consequences
of what is planned for the upcoming new CUPS.

As far as I currently understand it it seems
an underlying fundamental change in CUPS is
that CUPS will become basically "stateless".

The cupsd itself in the upcoming new CUPS will be stateful because
while it runs it keeps some state of the IPP printers it communicates with.
But when the cupsd is stopped it does not save its state persistently into some files.
When it is started it does not load a prior saved state from some persistent files.
So in the upcoming new CUPS there will be no longer be a persistently stored state.
The cupsd configuration files specify how cupsd behaves but do not store CUPS' state.
Also files as in #2
specify an ascpect how cupsd behaves but do not specify some part of CUPS' state.

In contrast in current CUPS there is a persistently stored state
mainly /etc/cups/printers.conf and the /etc/cups/ppd/queue_name.ppd files
and when current cupsd is stopped it saves its current state to files
and when current cupsd is (re)started it (re)establishes its state
according to a prior saved state from such persistent files.
Initially such a "prior saved state" could be provided by an admin.

If my understanding is right that CUPS will become basically "stateless"
as I described above then a consequence is that the upcoming new CUPS
can no longer be used to provide traditional print server functionality
in a managed environment.

With "traditional print server functionality in a managed environment" I mean:

An admin specifies his intended print server functionality persistently in config files
and the print server service process (i.e. cupsd) obeys what the admin specified.

With "persistently (in config files)" I mean that what the admin specified persists.
I.e. what the admin specified does not "magically" change "on its own".
I.e. only the admin specifies things.

E.g. when the admin specified a print queue with certain print queue specific settings
the print server will provide that particular functionality as specified by the admin
even if it is nonsense what the admin had specified - provided his specification
is not impossibe - i.e. basically as long as his specification is syntactically correct.
For example the admin can specify print queues for printers that do not exist
or he could even specify print queue specific things that will not work
for the printer that is associated with that print queue.

Bottom line from my point of view as far as I currently understand it:
When temporary queues have become the only technology in CUPS,
CUPS will be no longer usable as a print server.

[michaelrsweet]

@zdohnal WRT default options, the existing lpoptions support (which has been in CUPS since v1.1) is intended to provide the overrides you seek. The server/printer-side defaults are the base, then any local admin-defined defaults (in /etc/cups/lpoptions), then finally any per-user defaults (in ~/.cups/lpoptions). I don't want to add yet another way to specify/override default values as we have enough trouble getting applications to support the current API.

The profile mechanism should be hooked into the client side code (so libcups, not cupsd) and should transparently show up in the output from cupsGetDests and cupsEnumDests so that applications don't need to change. Just as for printers discovered via DNS-SD, a temporary queue will be created to handle spooling and filtering the first time you print to a profile printer.

[michaelrsweet]

@jsmeix So if you look at my most recent CUPS slide deck, one of my proposed future architectural changes is to split cupsd into two services - a per-user "local server" that handles the normal printing tasks and a system-wide "sharing server" that would provide persistent queues, ACLs, accounting, etc. Most users would never install the sharing server.

[jsmeix]

@michaelrsweet
thank you for the prompt reply.
It is much appreciated.

I had read
https://ftp.pwg.org/pub/pwg/liaison/openprinting/presentations/cups-plenary-may-2021.pdf
and I had noticed the CUPS 3.0 Local Server and Sharing Server parts
but I did not see the whole idea behind - in particular I did not see that
the CUPS 3.0 Sharing Server would still provide persistent queues.
Somehow the "Configuration similar to existing cupsd" did not trigger
the right things in my brain.

So as far as I now understand things:

The CUPS 3.0 Local Server basically provides what normal end-users usually need
in particular automated setup of what is needed to "just print" on IPP (Everywhere) printers.

The CUPS 3.0 Sharing Server basically provides what admins in a managed environment need
in particular persistent queues where admins can specify print queue specific things
as they need it in their particular environment.

[michaelrsweet]

@jsmeix Right. And since the sharing server doesn't have to deal with drivers it can (hopefully) end up providing a lot better support for accounting and other enterprise-y features... :)

[zdohnal]

@jsmeix

Bottom line from my point of view as far as I currently understand it:
When temporary queues have become the only technology in CUPS,
CUPS will be no longer usable as a print server.

Before I was corrected by @michaelrsweet in the next comment, I thought there will be no permanent queues in CUPS model in the future, so I thought additional features for profiles will be needed to cover the functionality, which was provided by permanent queues.
The original idea behind this issue was to cover use cases which aren't on the same LAN or are done by permanent queues right now with new, future CUPS compatible ways, so CUPS can be used as a print server even after removals of legacy technologies - meaning we would have not removed the old until the new is implemented and tested.
With my mistake corrected, I'll update the issue description to match up with the original idea - getting devices outside of LAN.

[zdohnal]

@michaelrsweet

@zdohnal WRT default options, the existing lpoptions support (which has been in CUPS since v1.1) is intended to provide the overrides you seek.

Aha, ok - so I've tried:

• using lpoptions for setting a default option from PPD to different value by:

$ lpoptions -p <temp_queue> -o <option>=<value>

, the option was applied and a permanent queue was created (so GTK will now get two same queues again :( ...), but I can see the correct default option for both.

The questions are:

1. Will be permanent queues available in CUPS 3.0 Local Server too, since changing default options depends on it?
2. Will be CUPS Web UI available in CUPS 3.0 Local Server? To have a GUI way how to set the defaults.

• using lpadmin to set job options associated with the queue:

$ lpadmin -p <temp_queue> -o <option>-default=<value>

This way, a normal permanent queue would change the option's value listed by lpoptions -p <queue>, but if we do it on temp queue, the temp queue will become permanent, but with invalid URI and the option isn't applied.:

$ lpadmin -p HP_LaserJet_M1536dnf_MFP_42307C_ -o copies-default=2
$ lpstat -v
device for HP_LaserJet_M1536dnf_MFP_42307C_: ///dev/null
$ lpoptions -p HP_LaserJet_M1536dnf_MFP_42307C_
device-uri=ipp://HP%20LaserJet%20M1536dnf%20MFP%20(42307C)._ipp._tcp.local/ Duplex=DuplexNoTumble printer-info='HP LaserJet M1536dnf MFP (42307C)' printer-location printer-make-and-model='HP LaserJet M1536dnf MFP' printer-type=83886100

Is it expected or is it a bug?

I don't want to add yet another way to specify/override default values as we have enough trouble getting applications to support the current API.

As permanent queues will not go away, there is no need for applications to create profiles, so no need for adding new operations.

The profile mechanism should be hooked into the client side code (so libcups, not cupsd)

I mentioned cupsd because that was the place where the proposed operation was supposed to be implemented.

and should transparently show up in the output from cupsGetDests and cupsEnumDests so that applications don't need to change. Just as for printers discovered via DNS-SD, a temporary queue will be created to handle spooling and filtering the first time you print to a profile printer.

Agreed.

[michaelrsweet]

@zdohnal OK, so I think there are still some misconceptions about this.

First, using lpoptions to set default options should definitely not be creating a permanent queue (or converting a temporary queue into a permanent one) - lpoptions is all client-side and should not affect server state. Moreover, lpoptions has dealt with temporary queues since the beginning when cupsd would create an delete local queues via the old CUPS browsing mechanism, so there is NO requirement for lpoptions to work with permanent queus. So if you are seeing permanent queues getting created, that is a bug (and please file it separately...)

Second, I hope I've made it clear that the local server is supposed to be very lightweight with no permanent queues, no web interface, and very little that can be configured. It is, by design, a transient thing that comes and goes only to handle whatever print jobs the user sends to a printer or print server. Printers and servers that are setup via profiles contribute to the list of potential destinations for the user to pick but queues for them are only created as needed and destroyed when not. Think of them like you are sending an email to someone - you don't keep a persistent connection to a particular mail server on the chance you might send an email to it, you connect as needed based on the messages being sent.

Using lpadmin to change the default option value doesn't convert a temporary queue into a permanent one, although if you specify "-o printer-is-temporary=false", it will. I'm not sure why you are seeing what you are seeing unless cupsd is going away between the lpadmin and lpoptions commands - it isn't what I would expect, but then there is no place for the default values to go when it is a temporary queue so I am not surprised either...