Change User-Agent to pymyq

[hstern]

Myq began blocking the User-Agent: "okhttp/3.10.0" header on 16/12/2020. This pull request changes the User-Agent to pymyq and fixes an UnboundLocalError that confused the issue.

[dale3h]

I can confirm by patching the homeassistant container on Home Assistant OS, this fixes PR the issue:

docker exec -it homeassistant sed -i 's/okhttp\/3\.10\.0/pymyq/g' /usr/local/lib/python3.8/site-packages/pymyq/api.py

[towerhand]

We have a lot of issues with the user agents, there's this PR in the sleepyq repo: technicalpickles/sleepyq#17

I'm not a dev but could something like this be implemented to myq?

[GaryOkie]

Homebridge devs updated their myq plugin and have already solved this issue for all their users. They did not hardcode a new user-agent mouse for the Chamberlain cat to chase...

Fixed, hopefully for the long-term now in v2.3.6. We now generate random user agent strings at startup. That should work until myQ decides to rejigger the API further.

Now I don't know if they are using this same pymyq or something else, but the point is this is a great idea.

hjdhjd/homebridge-myq#200 (comment)

[thedeany]

Now I don't know if they are using this same pymyq or something else

Nope, they have their own implementation of the API. Here's the line where they're setting a random user agent.

I don't know much about Python, but after a quick Google search it seems we could achieve the same thing like so:

''.join(random.SystemRandom().choice(string.ascii_uppercase + string.ascii_lowercase + string.digits) for _ in range(20))

I tested this out by importing string and random in api.py and setting DEFAULT_USER_AGENT to the above code. After a restart of HA, it seems to work. Again, I'm sure there's probably a better way to go about it. But if this code is fine, I'm happy to submit a PR.

[jsight]

Do we know what user agent the official client uses?

[dseven]

Do we know what user agent the official client uses?

Which official client? Back in June (#44) I determined (by using a MITM proxy, IIRC) that the Android client was specifying "okhttp/3.10.0". That got us another 6 months. Whatever we change it to, they could block it next week. If I get time, I might try to figure out how to do the MITM proxy thing again.

[hstern]

@dseven FWIW it worked fine without a User-Agent header at all.

[mikenabhan]

Do we know what user agent the official client uses?

Which official client? Back in June (#44) I determined (by using a MITM proxy, IIRC) that the Android client was specifying "okhttp/3.10.0". That got us another 6 months. Whatever we change it to, they could block it next week. If I get time, I might try to figure out how to do the MITM proxy thing again.

I will get this for iOS later today.

[jsight]

Do we know what user agent the official client uses?

Which official client? Back in June (#44) I determined (by using a MITM proxy, IIRC) that the Android client was specifying "okhttp/3.10.0". That got us another 6 months. Whatever we change it to, they could block it next week. If I get time, I might try to figure out how to do the MITM proxy thing again.

That's a fair point. Although if they are still using "okhttp/{version}" it may be worth following along as they may just be updating as they update their library dependencies. But you are right that there's no gauarantee that they won't start using some other approach to pretend to improve security.

[mikenabhan]

Not great news, I think they are using certificate pinning on iOS. I can't mitm the myq app and all requests just fail.

[rockford622]

So how do I implement a temporary change (by changing the User Agent as discussed in this thread) in my Home Assistant running on a Raspberry PI? I have SSH installed and have browsed the directory structure, but I don't seem to have the same paths as discussed in this thread. It's really annoying that Chamberlain intentionally changes their API to disable HA and other home automation platforms. It would be really nice if we could just communicate directly with the garage door opener.

[Landshark77]

So how do I implement a temporary change (by changing the User Agent as discussed in this thread) in my Home Assistant running on a Raspberry PI? I have SSH installed and have browsed the directory structure, but I don't seem to have the same paths as discussed in this thread. It's really annoying that Chamberlain intentionally changes their API to disable HA and other home automation platforms. It would be really nice if we could just communicate directly with the garage door opener.

This didn’t work ?

docker exec -it homeassistant sed -i 's/okhttp/3.10.0/pymyq/g' /usr/local/lib/python3.8/site-packages/pymyq/api.py

[rockford622]

So how do I implement a temporary change (by changing the User Agent as discussed in this thread) in my Home Assistant running on a Raspberry PI? I have SSH installed and have browsed the directory structure, but I don't seem to have the same paths as discussed in this thread. It's really annoying that Chamberlain intentionally changes their API to disable HA and other home automation platforms. It would be really nice if we could just communicate directly with the garage door opener.

This didn’t work ?

docker exec -it homeassistant sed -i 's/okhttp/3.10.0/pymyq/g' /usr/local/lib/python3.8/site-packages/pymyq/api.py

I don't use it in a docker container, I use it on a Raspberry PI. It doesn't even have that /usr/local/lib/python3.8 directory. I don't have any problem editing a file, if I can find it.

[rct]

This didn’t work ?

docker exec -it homeassistant sed -i 's/okhttp/3.10.0/pymyq/g' /usr/local/lib/python3.8/site-packages/pymyq/api.py

That won't work - too many /. Lost the backslash hat is quoting the slash after okhttp

[thedeany]

Yeah, the / in okhttp/3.10.0 needs to be escaped. This should work:

docker exec -it homeassistant sed -i 's/okhttp\/3.10.0/pymyq/g' /usr/local/lib/python3.8/site-packages/pymyq/api.py

[ehendrix23]

Might it maybe be easier to update the manifest.json instead? That should then result in the newer version of pymyq being pulled upon restart.

docker exec -it homeassistant sed -i 's/"pymyq==.*"/pymyq==2.0.12/g' /usr/local/lib/python3.8/site-packages/homeassistant/components/myq/manifest.json

[rockford622]

Might it maybe be easier to update the manifest.json instead? That should then result in the newer version of pymyq being pulled upon restart.

docker exec -it homeassistant sed -i 's/"pymyq==.*"/pymyq==2.0.12/g' /usr/local/lib/python3.8/site-packages/homeassistant/components/myq/manifest.json

So back to my original question, how do I perform this update on a Raspberry PI that is not running HA in Docker?

[Conundrum]

Do we know what user agent the official client uses?

Which official client? Back in June (#44) I determined (by using a MITM proxy, IIRC) that the Android client was specifying "okhttp/3.10.0". That got us another 6 months. Whatever we change it to, they could block it next week. If I get time, I might try to figure out how to do the MITM proxy thing again.

looked at the apk with apktool, looks like it was switched to http/3.12.1

Old App: smali_classes2/m/a/f.smali: const-string v0, "okhttp/3.10.0"
Latest App: smali_classes2/m/g0/d.smali: const-string v0, "okhttp/3.12.1"

[dseven]

Interesting. My assumption (guess) all along is that they are blacklisting user-agent's as a way to force app users to update to the latest version (probably to reduce support load from people reporting issues with old versions), and not some great conspiracy to thwart our efforts.

[Conundrum]

yep that's my guess. Glad I didn't send the nasty-gram e-mail I was planning to send yesterday :)

[fireheadman]

this worked perfect (for now)... thanks for the sed command

[SteveOnorato]

I don't use it in a docker container, I use it on a Raspberry PI. It doesn't even have that /usr/local/lib/python3.8 directory. I don't have any problem editing a file, if I can find it.

@rockford622 Are you using "Home Assistant Operating System" (HassOS) on that Raspberry Pi? If so, you really do have Docker containers, and that /usr/local/lib/python3.8 directory is encapsulated within the homeassistant "core" container. However, the default SSH won't allow you to interact with Docker; you need to disable 'Protection mode" to be able to use the docker commands.

• Click on the 'SSH & Web Terminal add-on' in the Supervisor -> Dashboard
  • Disable 'Protection mode' and click RESTART.

Then you when you use SSH, you should be able to see the containers:
docker ps
and also navigate into the actual homeassistant "core" container:
docker exec -it $(docker ps -f name=homeassistant -q) bash

[rockford622]

I don't use it in a docker container, I use it on a Raspberry PI. It doesn't even have that /usr/local/lib/python3.8 directory. I don't have any problem editing a file, if I can find it.

@rockford622 Are you using "Home Assistant Operating System" (HassOS) on that Raspberry Pi? If so, you really do have Docker containers, and that /usr/local/lib/python3.8 directory is encapsulated within the homeassistant "core" container. However, the default SSH won't allow you to interact with Docker; you need to disable 'Protection mode" to be able to use the docker commands.

* Click on the 'SSH & Web Terminal add-on' in the Supervisor -> Dashboard
  
  * Disable 'Protection mode' and click RESTART.

Then you when you use SSH, you should be able to see the containers:
docker ps
and also navigate into the actual homeassistant "core" container:
docker exec -it $(docker ps -f name=homeassistant -q) bash

Yes, I am. I don't see how to disable Protection Mode from the SSH dashboard. This is what my SSH dashboard looks like. There is nothing under the Configuration section either that relates to Protection Mode.

[Landshark77]

I don't use it in a docker container, I use it on a Raspberry PI. It doesn't even have that /usr/local/lib/python3.8 directory. I don't have any problem editing a file, if I can find it.

@rockford622 Are you using "Home Assistant Operating System" (HassOS) on that Raspberry Pi? If so, you really do have Docker containers, and that /usr/local/lib/python3.8 directory is encapsulated within the homeassistant "core" container. However, the default SSH won't allow you to interact with Docker; you need to disable 'Protection mode" to be able to use the docker commands.

* Click on the 'SSH & Web Terminal add-on' in the Supervisor -> Dashboard
  
  * Disable 'Protection mode' and click RESTART.

Then you when you use SSH, you should be able to see the containers:
docker ps
and also navigate into the actual homeassistant "core" container:
docker exec -it $(docker ps -f name=homeassistant -q) bash

Yes, I am. I don't see how to disable Protection Mode from the SSH dashboard. This is what my SSH dashboard looks like. There is nothing under the Configuration section either that relates to Protection Mode.

install the "SSH & Web Terminal" add-on. You're running a different SSH add-on.

[dseven]

Guys, you've totally hijacked a pull request that fixes a problem in the pymyq code, and turned it into a "how do I hack the python code used by various types of home assistant installations?"

If you don't know how to hack the fix in, I suggest waiting a few days until the various components (pymyq, homeassistant core, docker containers, homeassistant OS, etc.) pick up the fix, then update using the officially supported mechanism applicable to your installation.

[rockford622]

I don't use it in a docker container, I use it on a Raspberry PI. It doesn't even have that /usr/local/lib/python3.8 directory. I don't have any problem editing a file, if I can find it.

@rockford622 Are you using "Home Assistant Operating System" (HassOS) on that Raspberry Pi? If so, you really do have Docker containers, and that /usr/local/lib/python3.8 directory is encapsulated within the homeassistant "core" container. However, the default SSH won't allow you to interact with Docker; you need to disable 'Protection mode" to be able to use the docker commands.

* Click on the 'SSH & Web Terminal add-on' in the Supervisor -> Dashboard
  
  * Disable 'Protection mode' and click RESTART.

Then you when you use SSH, you should be able to see the containers:
docker ps
and also navigate into the actual homeassistant "core" container:
docker exec -it $(docker ps -f name=homeassistant -q) bash

Yes, I am. I don't see how to disable Protection Mode from the SSH dashboard. This is what my SSH dashboard looks like. There is nothing under the Configuration section either that relates to Protection Mode.

install the "SSH & Web Terminal" add-on. You're running a different SSH add-on.

Thank you, got the correct one now.

[rlust]

Yeah, the / in okhttp/3.10.0 needs to be escaped. This should work:

docker exec -it homeassistant sed -i 's/okhttp\/3.10.0/pymyq/g' /usr/local/lib/python3.8/site-packages/pymyq/api.py

This fixed it for me! Thanks!!!