Help! Instance syncronizing in a very weird way

Max_P, 11 months ago

Regarding your edit, I see a pattern here: you’re only seeing content from lemmy.ml and lemmy.world. Checking your instance’s federation list, you’re using the allowlist of only those and kbin.social. That means your instance is likely to be dropping anything from other instances, and everyone in this thread is from other instances.

Madiator2011, 11 months ago

How did you setup your reverse proxy? I had issues with federation on my server cause I messed up proxy :)

MajinBlayze, 11 months ago

ActivityPub appears to be very “eventually consistent” by design. There’s no guarantee that any given post or comment gets federated to you within a particular period of time.

jeena, 11 months ago

This is normal, because normally only stuff gets synced over:

1. When you search for that explicit post/comment, etc.
2. When you start subscribing everything after gets synchronized

Anyway, like others said, most of this weirdness fixes itself after a bit of time, the next day you should be having a fairly normal experience with the exception of the “All” because if you’re the only one on the server your Subscribed and All are exactly the same.

XTornado, 11 months ago

Wait, this means All is based on the instance users subscriptions? Or I am misunderstanding?

dis_honestfamiliar, 11 months ago

Asking the real questions.

das, 11 months ago

Yes, “All” shows all the communities users are subscribed to on your instance (since communities don’t federated with an instance until someone on that instance subscribes to it)

infamousbelgian, 11 months ago

To “fix” all, you can run a bot. Check !lsbsupport

tal, 11 months ago

I suspect that it works like the leafnode Usenet server did.

A full Usenet feed is a lot of traffic.

Leafnode would only download or update a newsgroup's contents when first requested by a client. But once it did so, it would store that data and make it available to other clients. It kept bandwidth requirements reasonable for Usenet servers with a small number of users.

The idea here is presumably aimed at scaling -- to basically try to only download what your users want, but once it comes down for one, to let all the others use it. Optimizes for your instance's bandwidth.

Max_P, 11 months ago (edited 11 months ago)

How long has it been? If you just spun it up, what you’ve seen is just the initial pull of the content. As you subscribe, all new content will get pushed to you, but the old content never backfills.

Check your NGINX logs, you should see a bunch of POST requests to /inbox and status of 200 that looks like this:

<pre style="background-color:#ffffff;">
<span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:16 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span><span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:16 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span><span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:17 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span><span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:17 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span><span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:18 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span><span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:19 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span><span style="color:#323232;">54.36.178.108 - - [13/Jul/2023:01:10:19 -0400] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.18.2; +https://lemmy.ml"
</span>

Included a bunch, as you can see they should be coming in quite frequently, basically everytime someone does something on the remote instance.

Make sure you’re subscribed to the community as well

Basic curl testing seems to rule out common routing problems, so check the lemmy logs as well just in case.

manitcor, 11 months ago

first day my instance was all over the map syncing, there are a few general mismatch bugs, a couple are caching. The big one is posts on one instance not propagating to all other instances, in particularly the home instance where the thread started.