I'm not sure how lemmy or kbin handle instance-hosted media links -- whether they import the media and redirect the link, or whether they point to the original media object -- but otherwise, yes.
There are ways to access other websites directly from within a given website -- iframes and the like -- but that's not what happens here. Each website is independent of each other, and all text is locally hosted in your instance's database.
There are also (limited) copies of user profiles all over the place -- if you click on my username, for instance, you'll be taken to lemmy.world/u/[email protected]. That's a local lemmy.world user address, even though I'm not on lemmy.world. I can't login to that account -- it's either credentialless, or has randomized credentials -- but it exists. And by going there, you get to see what lemmy.world knows about my activity across the fediverse. Without ever leaving lemmy.world.