[Feature]: Multiple parallel applications with same refresh-file cause exceptions

[paulvha]

Describe the Feature

Now that the refresh_token is set with a device-ID and can be saved there is a new aspect to take into account.
Multiple applications running in parallel using the same refresh-token file cause application to exit with exception error
I am not sure this is an enhancement or a warning with sharing of user experience. I run into this issue today and maybe save time for others.

Example:
background
With each device login a new unique device-ID is assigned for that login. You can do multiple device logins in parallel. The home-installation information as such can be accessed through different device-ID at the same time.

Situation description with saved refresh-token
Once an application-A has started, the refresh_token is obtained and is stored in a file AND in the application-A. The refresh-token in the application-A is continued to be used during the lifetime of application-A. At each access-token refresh (typical every 10 min) also the refresh token is updated in the file and in the application.
The previous refresh-token is NOT valid anymore. If application-A is stopped and restarted it will obtain the latest refresh-token from the file. All that works fine.

Now assume application-A is running continuous in the background. We also have application-B that we run to the same local PyTado instance. Application-B will use the same refresh-token file and it will obtain the refresh-token from that file to obtain the needed access-token. In doing so it will update the refresh-token file AND use the refresh-token in the application-B. NOW as soon as application-A tries to refresh the access token, the application-A has a refresh token that is NOT valid anymore and will fail with an exception.

If we now try to restart application-A, while application-B is still running, the same issue will happen but now application-B will fail with exception.

This only happens if we save the refresh-token in a file and we use the SAME refresh-token file for another application that is running in parallel.

It works fine in parallel if the refresh-token is NOT used from the file as a new unique device-ID is obtained at every login. But then you have to login every time you start the application.

---

Why Is This Feature Useful?

Reduce to potential on exceptions happening on (background) processes.
In case you want to have ONE refresh-token file that can be shared amount multiple application. In that case it will reduce the number or login's and people do not need to remember which refresh-token file with which application

---

Proposed Solution

A application-level workaround is provide a different refresh_token file for each application, that forces a one-time login for a new device-ID and thus it can work in parallel on the same PyTado instance.

Alternatives Considered

enhancement could be

If wanting or needing to use the same refresh-token file across different application, a check should added to determine whether the modification time on the file is the same as when last saved by that application.
If not it should first re-load the refresh token file before performing a refresh.

---

Checklist

• [X ] I have searched the existing issues and discussions to ensure this is not a duplicate.
• [ X] I have provided a clear and concise description of my feature request.

Thank you for helping to improve PyTado! 🚀

[wmalgadey]

@paulvha thank you for your finding. I see these solutions:

1. the calling code makes sure, it uses a different file, per used application instance! ;)
2. we could just call the _load_token() method every time we need to refresh the token

You could still use option 1 in the calling code, but make sure, the file isn't empty. You could just wait for Application-A to do the device-authentication first (and save the token), and then copy this file for Application-B and use "his" token. AFAIK the refreh-tokens will not be invalidated when we create a new one.

• cons:
  • the refresh-token will invalidate, if we create a new refresh-token
• pros:
  • not possible, because of invalid refresh tokens!

For option 2 we could add the functionality, to always load the value from the token file, when we do the refresh-token-flow. I don't know, if it is a problem if we do this, instead of checking the modification time of the file, but this would add more "trouble" than simplicity (we have to store the last modification time :D).

• cons:
  • access filesystem every time the refresh-token is used
• pros:
  • simple change to PyTado

But your finding introduces another issue with concurrency and race conditions. What if you start both applications at the same time? Both will do a device authentication and both will create a different refresh-token for a different device id, and after a time, the refresh-token-file contains either refresh-token of device-A or of device-B.
This can only be solved, by introducing some type of locking so only one device/application will do the authentication-flow, while the other may just display a "wait for other app" information, or even better present the same "visit url xy.com" from first application.

• cons:
  • syncing has to be done right to really work
• pros:
  • best solution in regards to developer exeprience

@paulvha what do you think? What's should we add the effort to do some device-auth-flow-sync?

[wmalgadey]

@paulvha what do you think of the solution in #179? We could easily add a lock_token and unlock_token method or you could use a single token_manager in all apps!

[paulvha]

I am not sure a lock is necessary. It works in parallel so far using different device-logins / and thus different device-ID. / and different refresh-file or even without a refresh-file and to a repeat login. I have not been able to get it to fail, but maybe I am lucky so far. maybe we Mike from Tado can shed a light on that.

You could still use option 1 in the calling code, but make sure, the file isn't empty. You could just wait for Application-A to do the device-authentication first (and save the token), and then copy this file for Application-B and use "his" token. AFAIK the refreh-tokens will not be invalidated when we create a new one.

That will not work. A refresh-token looks to only valid for ONE refresh. So once used by either application-A OR application-B it will be invalid for another use.

• The best way is either use different refresh-token-file for each application OR
• reload from the token-file before doing a refresh as you propose without a need to check for modification time OR
• Of course one can decide not to store the refresh-token and login every time.

[wmalgadey]

@paulvha I changed my comment for option 1.

[paulvha]

also thinking about the lock-file.. Do you mean that the issue is NOT Tado or deviceId, but if applications access the same PyTado instance could it happen that Application-A does a request, gets suspended as his time-slice is over in a multitasking operating system, now Application-B gets active and does a different or same request. Could application-A not get the requested information (or different) because application-B was enabled in between.??

I don't know the insides of Python and classes well enough, could that happen ?

[wmalgadey]

A classic race condition could happen:

1. if both applications start the authentication process at the same time
2. if app-a started the authentication, but the user doesn't respond asap, then app-b has to wait for the authentication, but app-b doesn't know, that there is already an auth-process in progress...

We could try to add the "sync"-functionality to the token_manager in the PR, or add another abstraction to the auth-process and make it even more user friendly, by adding the complete auth-process-information (like URL and device ID) into the sync process, so all apps started simultaneously are getting the same URL and device authentication state from the auth-syncer-class.

[paulvha]

That is real condition that I could see happen. There are indeed multiple solutions and I would not directly have a preference. Maybe it is a "once in a million" case and is it enough to provide warnings, recommendations and alternatives about running 2 applications to the same PyTado instance. An alternative could also be to advice each applications run against his own PyTado instance.

[jujuinparis]

Hello

i was using pytado with jeedom and simple scripts.

each script doing an action so creating an instance of pytado, doing the request and that s all.

With current token management process, even if I store the current token on file, I am facing an issue : I can have scripts running in parallel. Each script can update the refresh token so I am ending up with invalid token.

I was thinking of :

• having a process responsible for updating the refresh token as well as access token every 10 min
• having my scripts that are only using refresh and access token stored in the file with no possibility to update, just use and get needed data.

is there a way to change the refresh token method to only be on demand, and store access token as well in the file so that oauth header is set with refresh and access from file on load with no call to refresh_token method ?

thanks

[paulvha]

The easiest way was described by Wolfgang in an earlier post : just do a reload from the refresh-token file (if it exists) before doing a refresh of the access token. It is an easy change in the http.py in the function _refresh_token() to add a call self._load_token() after the initial check whether it should update. I applied it and it works.. but I see also they are working on a more robust approach.

having my scripts that are only using refresh and access token stored in the file with no possibility to update, just use and get needed data
This will not work,

• If the access code would also be stored in the file, you need to keep track how long ago that was stored as the access code only has a life-time of 10 minutes
• A refresh-token can only be used ONCE within 30 days..

[wmalgadey]

access_token should never be stored and shared. They are only usable a short amount of time (10 minutes currently). I will update my token_manager idea, to include a token_sync_manger which can savely be used to use PyTado from multiple integrations on the same host (everything else would be overkill, I guess and not the idea behind client auth flow!)

[wmalgadey]

@paulvha and @jujuinparis I did create a token manager class, which could be used to sync refresh token and solve the device activation delay between different instances of the api.

-> PyTado/token_manager/device_token_manager.py

if you have time, I would like some feedback 😄

[paulvha]

I have added in the pull-request #179