When an instance goes down, all of the images on it go with it. there’s going to be a lot from lemm.ee, especially given its size.

  • mad_lentil@lemmy.ca
    link
    fedilink
    English
    arrow-up
    3
    ·
    12 hours ago

    Not a solution, but this is a great reminder to add descriptions/alt text to your images.

  • Admiral Patrick@dubvee.org
    cake
    link
    fedilink
    English
    arrow-up
    53
    ·
    edit-2
    2 days ago

    I’m gonna run some scripts on my instance’s database to look for posts that have lemm.ee pict-rs URLs and local thumbnails. Then update the url value to that of the local thumbnail_url. That’ll at least fix them on my instance where I have a copy.

    I won’t be doing that until after lemm.ee shuts down, otherwise the original values will federate back in if there’s any activity on the posts.

    For comment images or ones where my instance didn’t create a thumbnail, they’ll just have to be broken I guess - same goes for the community icons, though I suppose I could go ahead and get local versions of those and update the communities afterward. Prob also set those to “Mods only” in the database to keep local users from trying to post there.

    For user avatars, I’ll probably just clear those in the database so when looking back at old content from .ee, it won’t try to fetch them and timeout/404.

    There were less posts to worry about then, but for community icons and user avatars, that’s the same thing I did when kbin shut down.

    • Shadow@lemmy.ca
      link
      fedilink
      English
      arrow-up
      18
      ·
      edit-2
      2 days ago

      Mind empowering my laziness and sharing that query?

      I was also planning on bulk locking all their communities, to avoid any isolated islands of people that don’t realize the community is gone.

      • Admiral Patrick@dubvee.org
        cake
        link
        fedilink
        English
        arrow-up
        10
        ·
        edit-2
        2 days ago

        I’m still putting that together, but it’s basically something like:

        UPDATE post set url = thumbnail_url where thumbnail_url like 'https://dubvee.org/pictrs/image/%' and url like 'https://lemm.ee/pictrs/image/%'

        For the Kbin users to clear out their avatar/banners, I just did:

        UPDATE person set avatar=NULL, banner=NULL where instance_id=(select id from instance where domain='kbin.run')

        UPDATE community set icon=NULL, banner=NULL where instance_id=(select id from instance where domain='kbin.run')

        …and will update that for lemm.ee

        I’m working on a Bun/NodeJS script to download the avatars, icons, and banners for all the lemm.ee users/communities, upload them to my pictrs, and then update the DB with the new values referencing the local images.

        Prob gonna have to break that up into two steps so I can download and file the images before lemm.ee goes down and upload/assign them after (so they’re not overwritten by federation).

        Not sure if I’m gonna do that for comment / post body images. Basically, I just want the archived communities to look nice.

  • _haha_oh_wow_@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    20
    ·
    2 days ago

    Real answer: We probably won’t, just like every other website.

    Some of the big places might have their own copies saved but most instances can’t afford to operate like that.

  • Microw@lemm.ee
    link
    fedilink
    English
    arrow-up
    16
    ·
    2 days ago

    Lemm.ee had and still has strict limitations to image uploads, forcing its users to use external image providers like catbox.moe or postimages. So these images should all still be there.

    Granted, I’m not sure how the thumbnails for these externally hosted images work and if they’re exclusively stored on lemm.ee

  • Goretantath@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    arrow-down
    2
    ·
    2 days ago

    Images dont get saved locally on other instances if they host the content? Well that seems like a big flaw in “federation”…

    • kersploosh@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      34
      ·
      2 days ago

      I assume it’s an intentional trade-off by the devs to reduce the storage burden. If each instance is a complete mirror of all federated instances then hosting becomes that much more expensive.

    • drkt@scribe.disroot.org
      link
      fedilink
      English
      arrow-up
      16
      ·
      2 days ago

      Instances have 3 options. Save it locally, proxy it, or do nothing. Saving every federated image is not feasible for small instances. It’s not a fault of the protocol that instance operators don’t want to pay a thousand dollars a month for storage. Matrix has this problem, and look what happened to them: tiny instances are expensive to run, so don’t flourish.

      • schnurrito@discuss.tchncs.de
        link
        fedilink
        English
        arrow-up
        6
        ·
        2 days ago

        no, I agree with the above user that it would be better to federate images too, for at least the following reasons:

        • redundancy: see the OP
        • privacy reasons: images loaded from other instances inform the other instance that one is viewing those images, do you trust all of them?
        • censorship reasons: some lemmy instances are blocked in my workplace and I can never look at images from them when browsing at work; others may be blocked in entire countries

        If I’m not mistaken, Mastodon already does federate images to other instances, so it isn’t unheard of.

        • hendrik@palaver.p3x.de
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          1
          ·
          2 days ago

          I think they would need to find a way to address the problem first. Reportedly, these images have been a huge problem here on Lemmy. Several times now.

            • hendrik@palaver.p3x.de
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              2 days ago

              Same, same. I can’t verify it and I probably don’t want to. But I had people assure to me it happened.

              • scytale@lemmy.zip
                link
                fedilink
                English
                arrow-up
                4
                arrow-down
                1
                ·
                2 days ago

                That’s credit towards the admins who acted quickly. That’s why you’ll see instances defederate from an instance if their admin is not active and doesn’t take action against illegal content in a reasonable time, because it can quickly get out of hand. Now if content isn’t proxied and is always mirrored, not only will the admin have to defederate from the problem instance, but they will have to go and clean up whatever was copied on their instance as well.

                • hendrik@palaver.p3x.de
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  edit-2
                  2 days ago

                  I can’t remember the exact details, but I believe the attackers also targeted instances? So it’s not just that it happens with certain problematic instances, but everyone could have that uploaded to their media storage. And it can come from arbitrary places. I believe that adds to the problem. And it kind of requires to shut these things down for everyone. Or at least everyone except a few excellent hand-picked instances who cooperate closely, and the moderation tools actually work.

                  Yes, they’ve done an excellent job. I just wish they wouldn’t have to deal with these things.

                  (And I also think some of the child protection agencies should finally offer some open-source tool to scan content. Afaik there are still no image classifiers or hash tables I could use for my projects.)

    • Admiral Patrick@dubvee.org
      cake
      link
      fedilink
      English
      arrow-up
      5
      ·
      2 days ago

      Depending on how the admin has pict-rs (the “subsystem” it uses for media) configured, post images can create local copies /thumbnails for image posts. Some instances opt to do that, some don’t.

      Other images like ones posted a comments, user/community avatars/banners aren’t stored anywhere except their home instance.

      But yeah, even the limited caching has proven problematic if questionable/illegal content gets posted.

  • IninewCrow@lemmy.ca
    link
    fedilink
    English
    arrow-up
    8
    ·
    edit-2
    2 days ago

    I don’t really care about images on here all that much … I don’t mind them being lost if I lose my instance.

    Most of the images that are shared are basically ‘one off’ images for a quick gag or as part of a conversation specific to that moment. And often that conversation or interact is a ‘one off’ event as well. Which means it was all meaningful when it happened and when it was read the few times it was noticed over two or three years but then it is forgotten.

    If there is anything meaningful I come across that I think is important to me or something I want to keep a reference of or keep … I’ll save it or screenshot it and put it away in my own storage or data.

    Otherwise, I really don’t mind losing all those other images I had created over the years … they already did their work when people saw them at that moment.

    The only reason anyone would want to keep all that data is to archive it for historical reasons … or to monetize it by collecting a bunch of unique data, images connected to real people.