PSA: You can upload images to a Lemmy instance without anyone knowing

EDIT

TO EVERYONE ASKING TO OPEN AN ISSUE ON GITHUB, IT HAS BEEN OPEN SINCE JULY 6: github.com/LemmyNet/lemmy/issues/3504

June 24 - github.com/LemmyNet/lemmy/issues/3236

TO EVERYONE SAYING THAT THIS IS NOT A CONCERN: Everybody has different laws in their countries (in other words, not everyone is American), and whether or not an admin is liable for such content residing in their servers without their knowledge, don’t you think it’s still an issue anyway? Are you not bothered by the fact that somebody could be sharing illegal images from your server without you ever knowing? Is that okay with you? OR are you only saying this because you’re NOT an admin? Different admins have already responded in the comments and have suggested ways to solve the problem because they are genuinely concerned about this problem as much as I am. Thank you to all the hard working admins. I appreciate and love you all.

ORIGINAL POST

cross-posted from: lemmy.ca/post/4273025

You can upload images to a Lemmy instance without anyone knowing that the image is there if the admins are not regularly checking their pictrs database.

To do this, you create a post on any Lemmy instance, upload an image, and never click the “Create” button. The post is never created but the image is uploaded. Because the post isn’t created, nobody knows that the image is uploaded.

You can also go to any post, upload a picture in the comment, copy the URL and never post the comment. You can also upload an image as your avatar or banner and just close the tab. The image will still reside in the server.

You can (possibly) do the same with community icons and banners.

Why does this matter?

Because anyone can upload illegal images without the admin knowing and the admin will be liable for it. With everything that has been going on lately, I wanted to remind all of you about this. Don’t think that disabling cache is enough. Bad actors can secretly stash illegal images on your Lemmy instance if you aren’t checking!

These bad actors can then share these links around and you would never know! They can report it to the FBI and if you haven’t taken it down (because you did not know) for a certain period, say goodbye to your instance and see you in court.

Only your backend admins who have access to the database (or object storage or whatever) can check this, meaning non-backend admins and moderators WILL NOT BE ABLE TO MONITOR THESE, and regular users WILL NOT BE ABLE TO REPORT THESE.

Aren’t these images deleted if they aren’t used for the post/comment/banner/avatar/icon?

NOPE! The image actually stays uploaded! Lemmy doesn’t check if the images are used! Try it out yourself. Just make sure to copy the link by copying the link text or copying it by clicking the image then “copy image link”.

How come this hasn’t been addressed before?

I don’t know. I am fairly certain that this has been brought up before. Nobody paid attention but I’m bringing it up again after all the shit that happened in the past week. I can’t even find it on the GitHub issue tracker.

I’m an instance administrator, what the fuck do I do?

Check your pictrs images (good luck) or nuke it. Disable pictrs, restrict sign ups, or watch your database like a hawk. You can also delete your instance.

Good luck.

db0,
@db0@lemmy.dbzer0.com avatar

Note, my tools is the only solution that exists (currently) for this in regards to csam

github.com/db0/lemmy-safety

bmygsbvur,

Appreciate your work.

BreakDecks,

I hate how everything is a double edged sword, because this is now also the perfect tool for making sure your CSAM doesn’t trip the filter. Also, it uses CLIP so a simple obfuscation overlay would render it useless.

db0,
@db0@lemmy.dbzer0.com avatar

can you elaborate on what an obfuscation overlay is?

BreakDecks,

Any of filter or image processing technique that fools machine vision.

Example: sandlab.cs.uchicago.edu/fawkes/

At a high level, Fawkes “poisons” models that try to learn what you look like, by putting hidden changes into your photos, and using themn as Trojan horses to deliver that poison to any facial recognition models of you.

This could be done with any kind of image or detail, not just faces.

db0,
@db0@lemmy.dbzer0.com avatar

I don’t think random trolls like that would be be that sophisticated, but in any case we can deal with that once we get to that point.

hypelightfly,

Not hosting images is a far better solution, and also exists.

robotrash,

What’s the best away for be to disable the pictrs directory? Is there a setting to flip in my instance?

HKayn,
@HKayn@dormi.zone avatar

Your contributions to the Lemmy ecosystem are much appreciated 🙏🏼

d3Xt3r,

Or just disable image uploads completely. We got by on Reddit without any built-in image hosting functionality for over a decade, so Lemmy should be fine without it as well - especially considering that we don’t really have many image-heavy communities, besides the NSFW instances. I mean, storage costs money you know, and with Lemmy being run by volunteers, it makes even more sense to get rid of image hosting to save costs.

cwagner,

deleted_by_author

    •
    d3Xt3r,

    This can be easily implemented client-side, like how third-party Reddit clients have been doing for years, by uploading to the likes of Imgur. Shift the responsibility away from the Lemmy server and onto dedicated image hosts, who have the storage to spare, plus manpower/policies to deal with illegal content.

    cwagner, (edited )

    deleted_by_author

    •
    d3Xt3r, (edited )

    Desktop users exist

    So do Desktop tools like Flameshot, which can directly upload to image hosts and copy the URL to the clipboard which makes it easy to share images, and there also exists third-party Desktop web-clients such as Photon, which could be updated with that functionality as well. But with Lemmy itself being open source, it wouldn’t take much effort to modify the code to use a third-party image host.

    have a history of deciding to forbid hotlinking

    There are plenty of hosts which do allow hotlinking though, like imgbb.com

    history of suddenly deleting all (e.g. PhotoBucket) or some (e.g. Imgur) images .

    Not a big loss, IMO. Lemmy isn’t an image hosting nor an image-centric site, it’s a text-heavy forum at first instance, and anyone posting images are encouraged to provide text alts for the benefit of blind users, so images not persisting isn’t a big deal.

    If image persistence is really that important, there are other services which are better suited for that, such as Pixelfed. But in the first place, I wouldn’t rely on some random Lemmy server, which is vulnerable to DDoS and other attacks and could go down at any time (also why the importance on decentralization - no single instance is infallible). I mean, when there’s no guarantee that a Lemmy instance will even be there tomorrow, is there really a need to worry about image persistence?

    CaptainAniki,

    catbox exists

    CaptainAniki,

    I don’t have the pictrs container running on my instance.

    Corgana,
    @Corgana@startrek.website avatar

    💯 I really hope this is in the next update

    r00ty,
    @r00ty@kbin.life avatar

    I'm not using lemmy. But I was thinking of making a process to periodically scan the object storage and check for a reference to a post, comment etc and if none are found delete it. In most cases the images are deleted but sometimes they don't seem to be.

    Probably lemmy could have a similar process created.

    Swedneck,
    @Swedneck@discuss.tchncs.de avatar

    seems like the solution to this should be to automatically remove images that haven’t been posted, after like 3 minutes

    bmygsbvur,

    This is one way to solve it.

    cwagner,

    deleted_by_author

    •
    Kitikuru,

    The 3 minutes would only kick in if an image was uploaded but then never posted. So nobody would see it anyway in any case.

    This route would avoid the issue but also help save on space too.

    Racle,
    @Racle@sopuli.xyz avatar

    What happens if user spends over 3 minutes to write the post after uploading image?

    Would user create a post with broken image link? or would there be some kind of “call home” API call on create post page so image wouldn’t be removed? (which has risk that API call could be replicated by a bot)

    Kitikuru,

    That is a good point. Could potentially not upload the image until the post is created instead of at image choosing, which would also alleviate the issue. But I’m not sure how that would work across web and mobile clients.

    squiblet,
    @squiblet@kbin.social avatar

    I think that's the best solution. I can't see a reason any client couldn't upload the image when the post is submitted. Currently the uploader is some fancy javascript deal and it's unnecessary.

    Swedneck,
    @Swedneck@discuss.tchncs.de avatar

    Could allow for like one hour of keep-alive pings before it’s deleted and the client is told to notify the user of this

    Also: rate limits that gradually increase are good.

    NewDataEngineer,

    Write the post and then upload the image?

    CptMuesli,

    This could be handled by the client. Get the Ruleset for image uploads (max size, format, etc.), Validate the image within the client, only upload when the post is published.
    Then the delay between post and image only depends on your internet connection and the user can still take 3 hours to write a post.

    porkins,

    The solution is to base it upon the user session timer.

    russjr08,
    @russjr08@outpost.zeuslink.net avatar

    As in when you’d normally get automatically logged out? If so, I’m not sure that would work since Lemmy uses JWTs that don’t expire (or if they do, not for a very long time) it seems.

    Venat0r,

    Or make it like 1hr and don’t let the user know the url of the uploaded image until they post it, that way it wouldn’t be able to be shared or reported.

    squiblet,
    @squiblet@kbin.social avatar

    It's difficult to display an image without the client knowing the URL, but it would be possible to use a temporary URL that only works for that signed-in user.

    Venat0r,

    Store the image in memory, or in browser cache.

    JackbyDev,

    3 minutes is way too short. You could upload and it be gone by the time you post.

    pqdinfo,

    Or just mark the resource as private and only serve it to the user who created it until they associate a post with it.

    You would probably need a separate server to stage images like this, as your main image server probably shouldn’t have a login wall, which probably slightly complicates things but not badly.

    KIM_JONG_JUICEBOX,
    @KIM_JONG_JUICEBOX@lemmy.ml avatar

    Or you set a flag that says something like “incomplete image” and then only once user completes whatever operation by hitting “submit” do you then set it to complete.

    And maybe while an image is not yet complete, only the uploading user can view the image.

    squiblet,
    @squiblet@kbin.social avatar

    It would not be difficult to use SQL to delete any images that are not associated with a post or active as an avatar etc. So, set that to be run periodically and it would solve this problem.

    bmygsbvur,

    I’m not knowledgeable with SQL. If you know or if anyone knows how to fix it with a script or built into Lemmy, please share.

    squiblet,
    @squiblet@kbin.social avatar

    I haven't worked with Lemmy, but I certainly could craft a script to do that if I was familiar with the database structure. Perhaps I'll try installing it and running an instance. In the meantime, surely there's someone with an instance and SQL skills who could figure that out.

    relative_iterator,
    @relative_iterator@sh.itjust.works avatar

    Yeah it shouldn’t be hard at all. Just need to know the link between images and posts.

    Kaldo,
    @Kaldo@kbin.social avatar

    Isn't it more likely that paths are used to reference resources like images rather than a db fk?

    squiblet,
    @squiblet@kbin.social avatar

    Not familiar with Lemmy specifically, but usually in an app like this, while of course the files are stored on a filesystem, IDs and metadata are stored in the DB and associated with each other through relations. It seems in this case one way to express it would be 'don't delete every image that is associated with a valid post or in-use avatar, but delete everything else'.

    Take this random image for instance: https://lemmy.world/pictrs/image/ede63269-7b8a-42a4-a1fa-145beea682cb.jpeg
    associated with this post: https://lemmy.world/post/4130981

    Highly likely the way it works is there is an entry for post 4130981 that says it uses ede63269-7b8a-42a4-a1fa-145beea682cb, or an image table with a relation to the post table where an image entry (with whatever ID) that is ede63269-7b8a-42a4-a1fa-145beea682cb says it is related to post 4130981. Whatever the specifics, it would be possible.

    ExLisper,

    Post in lemmy can just store the URL and both system can have no idea about one another on the DB level. From what the post is saying, the image in pictrs is created before lemmy post gets its ID so I don’t think you can do what you’re describing. What you can do is:

    • iterate over all post created in the lst 24h
    • list all the links to pictrs in their content
    • list all images created in the last 24h
    • delete all images that are not in the first list

    You can change 24h to 1h and run it from cron.

    squiblet,
    @squiblet@kbin.social avatar

    That is essentially what I was saying. It does seem like it would take a scripting element in addition to SQL.

    ExLisper,

    Yes but very simple. The main issue is accessing the DB really. I will set up my own instance and try to figure it out.

    gencha,

    Checking every single image ID against all stored text blobs is not trivial. Most platforms don’t do this. It’s cheaper to just ignore the unused images.

    squiblet,
    @squiblet@kbin.social avatar

    Yeah, this is only if what OP was saying was a real legal threat, which I don’t think it is.

    Dirk,
    @Dirk@lemmy.ml avatar

    This is how it works. Since pictrs and Lemmy are two completely different applications (they even run in two different containers with two different databases) they do not communicate and tracking what images belong to what post or comment simply isn’t possible in the current state I guess.

    How come this hasn’t been addressed before?

    This is how the Fediverse works. There is so much bad practices, so much haphazardly implemented functionality and so much bad API documentation all over the place that I wonder why nothing has extremely exploded so far. We don’t even have proper data protection and everything is replicated to everywhere causing a shitload of legal issues all over the workd but no-one seems to care so far.

    Spiralvortexisalie,

    Sounds like the Internet Protocol I grew up with 😍

    danwardvs,

    This isn’t unique to Lemmy or haphazard coding. It’s a common technique to get pictures into Github READMEs this way. You’d create a PR, upload an image, copy the link, delete the PR, and then paste the link elsewhere on Github for use.

    eleanor,

    The difference between the Fediverse and a closed system like reddit is that it’s open and we’re privy to haphazardly implemented functionality and bad API documentation.

    I work on big closed source web apps for a living; they’re just as haphazard and badly documented, it’s just all closed.

    RegalPotoo,
    @RegalPotoo@lemmy.world avatar

    Yes, an app designed to receive user generated content will allow users to upload arbitrary content. This is not really an issue unique to Lemmy, and can be fixed by banning abusive users

    bmygsbvur,

    Except admins need to go through the database to catch it. Mods can’t catch it. Admins without access to the database or object storage can’t catch it.

    contextual_somebody,

    As far as I know, people don’t homebrew Reddit.

    squiblet,
    @squiblet@kbin.social avatar

    Sites like reddit could have about the same problem. You can post to some obscure profile and use reddit as image hosting and it might take quite some time before anyone notices. This is a little worse though because it wouldn't even be listed on a profile page.

    drkt,

    In theory also possible to just be a nuisance by filling out the instances available space? That sounds like it’s gonna get fixed one way or another.

    bmygsbvur,

    Yes - that’s possible.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • uselessserver093
  • Food
  • aaaaaaacccccccce
  • [email protected]
  • test
  • CafeMeta
  • testmag
  • MUD
  • RhythmGameZone
  • RSS
  • dabs
  • Socialism
  • KbinCafe
  • TheResearchGuardian
  • oklahoma
  • feritale
  • SuperSentai
  • KamenRider
  • All magazines
    •