@malte So you're seeing a meaningful difference between the view from a "tiny inactive server" vs something larger, but not necessarily something larger vs the whole fediverse. A matter of grade, then.
As it is today, we can't even say how large is the section of fedi its biggest individual server, mastodon.social (1.5M users out of 10.5M, according to fedidb.org) does not see. Could be marginal (assuming most active users get federated over there), could be very large (if significant sub-clusters not connected to mastodon.social exist - and some do, because mastodon.social is blocked by some servers).