Check Sync fails on RM2 running 3.3.2.1666

[BHSPitMonkey]

I've been running an rmfakecloud container for a few years (thanks for this!), and somewhere along the way I realized my RM2 running 3.3.2 stopped syncing. I'm able to successfully re-auth using a new code, but afterward sync does not work.

Symptoms

On the tablet, the "cloud offline" status bar icon appears. When I tap "Check sync" in storage settings, it waits 5 seconds and then gives "There are some issues with the cloud connection. Please try again later."

When this happens, no new logs are printed in the rmfakecloud container or in journalctl -f running on the tablet.

The only logs that do appear on the tablet are these warnings, posted every 30 seconds (independent of "Check sync", as in #291):

Jul 24 16:55:45 reMarkable xochitl[282]: 16:55:45.244 rm.network.notifications Notifications socket is not OK: UnconnectedState (checkIfShouldConnect /usr/src/debug/xochitl/override+gitAUTOINC+6a003d604f-r0/git/src/notifications/src/notifications.cpp:187)

Server and Client

I'm running rmfakecloud in Docker (not behind any reverse proxy, though I have also tried adding one with a WebSocket-friendly config just to see what would happen and nothing changed). I re-map the container's port 3000 to 3030 on the host (3000 was already used).

When I run rmfakecloudctl status on the tablet, I see:

Status: enabled (active)
Upstream server: http://192.168.1.101:3030 (working)

and the upstream status check shows up in the server container's logs:

rmfakecloud  | time="2025-07-24T16:59:15Z" level=info msg="Requested: test\n"                                                      
rmfakecloud  | time="2025-07-24T16:59:15Z" level=info msg="[GIN] 2025/07/24 - 16:59:15 | 200 |      79.293µs |      172.23.0.1 | GET      \"/service/json/1/test\""

I've tried rmfakecloudctl disable && rmfakecloudctl enable at various points just to force rmfakecloud-proxy to reconfigure itself, and this makes no difference (it succeeds every time).

When I open a document on the tablet, I do notice a flood of requests start coming through:

# Tablet journalctl:
Jul 24 17:11:34 reMarkable xochitl[282]: 17:11:34.192 rm.network.usertoken     Scheduled token update at 2025-07-24 20:06:31, 10496807ms from now (scheduleUpdate /usr/src/debug/xochitl/override+gitAUTOINC+6a003d604f-r0/git/src/network/src/usertoken.cpp:141)
Jul 24 17:11:34 reMarkable xochitl[282]: 17:11:34.192 rm.network.usertoken     UserToken: setting a new userToken ("eyJhbGciOiJIUzI1NiIs"...) (setUserToken /usr/src/debug/xochitl/override+gitAUTOINC+6a003d604f-r0/git/src/network/src/usertoken.cpp:73)

# Docker container:
rmfakecloud  | time="2025-07-24T17:12:25Z" level=info msg="[GIN] 2025/07/24 - 17:12:25 | 200 |     777.033µs |      172.23.0.1 | POST     \"/token/json/2/user/new\""
rmfakecloud  | time="2025-07-24T17:12:25Z" level=info msg="setting scopes: intgr screenshare docedit hwcmail:-1 hwc sync:default"

Asks

Are there any known problems using this firmware version with newer versions of rmfakecloud? If not, any suggested troubleshooting steps I should take next?

[Eeems]

What version of rmfakecloud are you running?

[BHSPitMonkey]

What version of rmfakecloud are you running?

Latest (v0.0.25)

[ArnoldDeRuiter]

Wanted to chime in, I also have the same issue and logs, journalctl and docker.
Also been running it for years, had to login again after I realized it stopped syncing as well.

Running RM2 software: 3.11.3.3 (rM Hacks v0.0.10)
Docker image: ddvk/rmfakecloud: v0.0.25

[nemunaire]

Could you test some older rmfakecloud releases to tell us which version it starts to malfunction from?

We don't test new releases with old remarkable software releases, but with your help, we can try to make it compatible.

Did you just update from 0.0.24 and it stops working? or did you use a previous release?

[ArnoldDeRuiter]

Yes, I was just pinning my docker to 0.0.24, and tested it just now.
It works again, it took a while of course, but it works.

Again, that is for me, running RM2 software: 3.11.3.3 (rM Hacks v0.0.10) anyways.
Not sure if this will fix things for BHSPitMonkey running 3.3.2 as well?

[nemunaire]

Thanks a lot for checking 🙏

The only change with 0.0.24 that could lead to this is cdb45df. Others commits are not relevant.

I forgot to mention in the release notes, but a new domain has to be added in /etc/hosts with 0.0.25: eu.tectonic.remarkable.com (see #364). Do you have it?

[ArnoldDeRuiter]

Huh, so I added the eu.tectonic.remarkable.com line to etc/hosts:

127.0.0.1	localhost

[...]
# rmfake_end
127.0.0.1 eu.tectonic.remarkable.com

And I just unpinned the rmfakecloud docker, updating it to 0.0.25 again.
I uploaded a PDF, and it seems to have synced just fine. 🙂

Considering the etc/hosts file is usually updated only by the proxy script, should it be updated, along with the manual documentation?

Hopefully BHSPitMonkey can confirm this works as well on their setup.

[ClementGre]

Same issue for me on RM2 3.11.2.5. rmfakecloud stopped working in v0.0.25, and the fix of @ArnoldDeRuiter worked for me, thank’s.

[BHSPitMonkey]

Unfortunately no improvement for me on my 3.3.2 device after trying the hosts file change; The same symptoms persist. I've also tried (making a backup and) rolling back the server to prior tags going back to v0.0.18, as well as manually installing the latest rmfakecloud-proxy release (instead of the older version in Toltec I'd been using).

One thing I've noticed since upgrading rmfakecloud-proxy is that my journalctl UnconnectedState errors (every 30s) are now accompanied by these TLS handshake errors (the specific local port changes each time it is logged):

Jul 28 18:26:19 reMarkable rmfakecloud-proxy[4201]: 2025/07/28 18:26:19 http: TLS handshake error from 127.0.0.1:39610: EOF
Jul 28 18:26:19 reMarkable xochitl[4216]: 18:26:19.759 rm.network.notifications Notifications socket is not OK: UnconnectedState (checkIfShouldConnect /usr/src/debug/xochitl/override+gitAUTOINC+6a003d604f-r0/git/src/notifications/src/notifications.cpp:187)

Do we know exactly what the "Check sync" button actually does (or should be doing), such that I could try to mimic its steps using other tools (curl/nc/etc.) to figure out what's missing?

[Eeems]

as well as manually installing the latest rmfakecloud-proxy release (instead of the older version in Toltec I'd been using).

Any chance you'll open a PR to get it updated in toltec? It's much faster to get things in when someone else opens a PR.

[BHSPitMonkey]

as well as manually installing the latest rmfakecloud-proxy release (instead of the older version in Toltec I'd been using).

Any chance you'll open a PR to get it updated in toltec? It's much faster to get things in when someone else opens a PR.

If you mean me, I'm willing to look into how to do that sometime (though I'm not a maintainer and don't know if the newer versions are fully safe to promote). But upgrading to the newer release did not solve any of the problems I mentioned—I just included it to detail what I've tried so far.

[nemunaire]

I use openssl s_client -host internal.cloud.remarkable.com -port 443 on the tablet (without curl/toltec).
If you have curl, you can try curl https://internal.cloud.remarkable.com: you should see the rmfakecloud home page, with no certiticate error.

When I open a document on the tablet, I do notice a flood of requests start coming through:

This seems similar to #262. But it was related to a missing entry on my side.

Most probably, the JWT is wrong, has missing expected fields. I see this commit that can be the root cause: d94e36c.
Could you test v0.0.14, before this commit, it should work, and v0.0.15, it should not work.

[BHSPitMonkey]

I use openssl s_client -host internal.cloud.remarkable.com -port 443 on the tablet (without curl/toltec).

This works.

If you have curl, you can try curl https://internal.cloud.remarkable.com: you should see the rmfakecloud home page, with no certiticate error.

When I try that, I run into:

reMarkable: ~/ curl -vv https://internal.cloud.remarkable.com
23:52:25.297286 [0-0] * TLSv1.3 (OUT), TLS handshake, Client hello (1):
23:52:25.390544 [0-0] * TLSv1.3 (IN), TLS handshake, Server hello (2):
23:52:25.393465 [0-0] * TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
23:52:25.393780 [0-0] * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
23:52:25.394162 [0-0] * TLSv1.3 (IN), TLS handshake, Certificate (11):
23:52:25.396614 [0-0] * TLSv1.3 (OUT), TLS alert, unknown CA (560):
23:52:25.396816 [0-0] * SSL certificate problem: self-signed certificate in certificate chain
curl: (60) SSL certificate problem: self-signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.

With some LLM-guided troubleshooting I was able to root-cause that to the fact that curl is using /opt/etc/ssl/certs/ca-certificates.crt rather than the one rmfakecloudctl updates via update-ca-certificates (/etc/ssl/certs/ca-certificates.crt, outside /opt). I'm not sure if this is actually a problem for xochitl, or just a red herring?

Could you test v0.0.14, before this commit, it should work, and v0.0.15, it should not work.

No change after switching the backend to v0.0.14; The same symptoms persist 🤔 I'm becoming more convinced this is some kind of client-side issue

[witchof0x20]

Just noting here that uninstalling rmfakecloud-proxy from toltec and installing the latest version from https://github.com/ddvk/rmfakecloud-proxy manually fixed this issue for me.

RM1, 3.3.2.1666

[chosso]

uninstalling rmfakecloud-proxy from toltec and installing the latest version manually

worked for me too, thanks! (RM2, 3.3.2.1666)