Does lemmy have any communities dedicated to archiving/hoarding data?

    • ulterno@programming.devEnglish
      4·
      1 day ago

      debian stable became the go to distro for long term usage in case our FOSS support structure goes haywire due to wars

  • Dagamant@lemmy.world
    15·
    2 days ago

    I would add in some rom collections and book repositories as well. The whole library of Nintendo games is under a gig and would go a long way for entertaining people.

  • GreenKnight23@lemmy.world
    411·
    2 days ago

    I have been archiving Linux builds for the last 20 years so I could effectively install Linux on almost any hardware since 1998-ish.

    I have been archiving docker images to my locally hosted gitlab server for the past 3-5 years (not sure when I started tbh). I’ve got around 100gb of images ranging from core images like OS to full app images like Plex, ffmpeg, etc.

    I also have been archiving foss projects into my gitlab and have been using pipelines to ensure they remain up-to-date.

    the only thing I lack are packages from package managers like pip, bundler, npm, yum/dnf, apt. there’s just so much to cache it’s nigh impossible to get everything archived.

    I have even set up my own local CDN for JS imports on HTML. I use rewrite rules in nginx to redirect them to my local sources.

    my goal is to be as self-sustaining on local hosting as possible.

    • Foster Hangdaan@lemmy.hangdaan.com
      3·
      1 day ago

      Everyone should have this mindset regarding their data. I always say to my friends and family, “If you like it, download it.”. The internet is always changing and that piece of media that you like can be moved, deleted, or blocked at any time.

  • Retro_unlimited@lemmy.world
    33·
    2 days ago

    I also recommend downloading “Flashpoint archive” to have flash games and animations to stay entertained.

    There is a 4gb version and a 2.3TB version.

  • pyrflie@lemmy.dbzer0.com
    79·
    2 days ago

    Welcome to datahoarders.

    We’ve been here for decades.

    Also follow 3-2-1 people. 3 Backups, 2 storage mediums, 1 offsite.

      • wurstgulasch3000@feddit.org
        21·
        2 days ago

        You know there’s only two kind of people, those who do backups and those that haven’t lost a hard drive/data before. Also: raid is no backup

        • Valmond@lemmy.world
          3·
          2 days ago

          Still remember the PSU blast taking out my main drive plus my backup drive in like 2001. I thought I was so good because I at least had a backup 😑. Those were the days 🤷🏻‍♀️

        • Isthisreddit@lemmy.worldEnglish
          103·
          2 days ago

          Conservatives hate knowledge, learning is toxic to them. Also the people who start with burning books usually end up burning people eventually

          • TangledHyphae@lemmy.world
            318·
            2 days ago

            Removing books about sucking cum out of anuses from public schools isn’t really “burning books.” You can still buy them whenever you want, just not putting them in taxpayer funded schools with children.

            EDIT: Had to add some details of the “books being burned [but really just removed from public school]”:

            During public comment, one woman read a passage from “Yolo” by Lauren Miracle which is found in Freedom High School.

            “I climbed onto of him and started kissing him in a way that said very clearly here I am, I’m ready to have sex,” the speaker read.

            Another title, “Anatomy of a Single Girl” by Daria Snadowsky, was also read by a speaker.

            “Guy tries rubbing my clitoris with his fingers, he wiggles his pelvis back and forth,” another woman read from the book.

            “This is ridiculous that this school – any school – has this book,” the woman said to the board.

            Julie Gebhards, the woman seen in the first video of our story, is a Hillsborough County mom of six children.

            Gebhards read an excerpt from the book “Invisible Monsters Remix” by Chuck Palahniuk. According to the district’s online book library, the title is found in Steinbrenner High School.

            “He shoots his load, and then plants his mouth on your anus and sucks out his own warm sperm, plus whatever lubricant and feces are present. That’s felching. It may or may not, I add, include kissing you to pass the sperm and fecal matter into your mouth,” Gebhards said.

            • WoodScientist@lemmy.world
              10·
              2 days ago

              We should ban mention of Christianity in public. We should also make it illegal for anyone to teach their children Christianity. Practicing Christians should be declared mentally ill, and if they practice their faith in front of children, they should be put on the sex offender registry.

              These freaks actually put giant statues of a naked bleeding man up on full public display in buildings. And they believe the most holy book in the world is one that features incest, murder, rape, genocide, and often fully endorses these horrors. Their main ritual is a form of public ritual cannibalism.

              Christians are too dangerous to be allowed near children.

            • kadu@scribe.disroot.org
              20·
              2 days ago

              The comment you’re replying to didn’t mention one specific book. You did to try and portray this as some noble cause, but even books such as Fahrenheit 451 and To Kill a Mockingbird have been banned by conservatives, and they most definitely aren’t about “sucking cum out of anuses” as you so dumbly put it.

              Nice attempt, but this type of dodging around never worked and will never work.

              • lightnsfw@reddthat.com
                3·
                2 days ago

                As if kids aren’t finding shit way worse on the internet on a daily basis. Well… maybe not felching that’s pretty vile. But still.

            • fibojoly@sh.itjust.works
              4·
              2 days ago

              Oh no! What have you done! Now, I want to go try felching because I saw a message about it online with no context and I just. Have. To. Try. It.
              … Oh wait, no. No, I don’t. Pfew!

              So what was your point again?

  • Maroon@lemmy.world
    32·
    2 days ago

    I thought the whole point of torrenting was to decentralise distribution. I use torrents to get my distros.

    In my own little bubble, I thought that’s how most people got their distro.

  • Gerowen@lemmy.worldEnglish
    41·
    2 days ago

    Neither are that bad honestly. I have jigdo scripts I run with every point release of Debian and have a copy of English Wikipedia on a Kiwix mirror I also host. Wikipedia is a tad over 100 GB. The source, arm64 and amd64 complete repos (DVD images) for Debian Trixie, including the network installer and a couple live boot images, are 353 GB.

    Kiwix has copies of a LOT of stuff, including Wikipedia on their website. You can view their zim files with a desktop application or host your own web version. Their website is: https://kiwix.org/

    If you want (or if Wikipedia is censored for you) you can also look at my mirror to see what a web hosted version looks like: https://kiwix.marcusadams.me/

    Note: I use Anubis to help block scrapers. You should have no issues as a human other than you may see a little anime girl for a second on first load, but every once and a while Brave has a disagreement with her and a page won’t load correctly. I’ve only seen it in Brave, and only rarely, but I’ve seen it once or twice so thought I’d mention it.

    • trashboat@midwest.socialEnglish
      41·
      2 days ago

      I rarely get bounced by Anubis, but oddly enough it has happened to me a couple times in FF, I suspect it’s the fingerprinting resistance settings that cause this to happen? Hasn’t happened in a while though

    • clif@lemmy.world
      8·
      2 days ago

      It makes me really happy that people can say “500gb … not too much of an ask” these days.

      • blind3rdeye@aussie.zone
        2·
        1 day ago

        Well we are talking about the greatest repository of human knowledge ever created. So we can afford to spend a little on it at least.

    • BwahFox@lemmy.world
      4·
      2 days ago

      i know this because i actually do this. its more like ~300gb of space but its better to have even more just in case

    • clif@lemmy.world
      41·
      3 days ago

      Last time I updated it was closer to 120GB but if you’re not sweating 100 GB then an extra 20 isn’t going to bother anyone these days.

      Also, thanks for reminding me that I need to check my dates and update.

      EDIT: you can also easily configure a SBC like a Raspberry Pi (or any of the clones) that will boot, set the Wi-Fi to access point mode, and serve kiwix as a website that anyone (on the local AP wifi network) can connect to and query… And it’ll run off a USB battery pack. I have one kicking around the house somewhere

      • techwithjake@sh.itjust.works
        14·
        3 days ago

        Just built one of those using Dietpi as the OS and NVME M.2 for the storage. I have many different ZIMs and running different services and only using about 270GB.

        Works great for offline use. Probably should add an ISO or 2 as well.

        • clif@lemmy.world
          2·
          2 days ago

          What other services are you running?

          @fmstrat@lemmy.world asked what else I was running in a sibling comment to yours and I didn’t have an answer because I’m not… yet : )

          • techwithjake@sh.itjust.works
            7·
            2 days ago

            DietPi makes it dead simple to run most of these things as their “software suite” is pretty robust and simple to setup.

            For “user facing” applications:

            • Homer Dashboard as the landing page when going to the .local address in a browser
            • Kiwix for the ZIMs
            • Hedgedoc for personal note taking/wiki
            • Lychee photos for a very lightweight photo album maker/viewer for keepsake photos.

            For “admin side” stuff:

            • Portainer to manage the containers/stacks
            • Watchtower to auto-update the containers while they’re still network connected
            • Transmission daemonized to download and seed the ZIMs or anything else non-pirate related
            • Use jojo2357’s ZIM updater to auto-update ZIMs via cron job while they’re still network connected
            • DietPi-Dashboard as an all-in-one dashboard to monitor and control the RPi from a web interface. (Yeah I know I can do everything SSH’ing in but I’m lazy.)
            • File Browser just in case I want other people to have access to files but since it’s in maintenance mode and I’m unsure I want others to have access, might strip it out

            I try to use containers from LinuxServer.io whenever possible. Mostly just cause it’s what I do on my main server.

            I’m still looking at adding/removing things as I get more time to sit down but I’m pretty happy with it’s current state.

      • Fmstrat@lemmy.world
        2·
        2 days ago

        Do you recommend adding anything else to it?

        For instance, OSM maps?

        I’ve been thinking about running the Kiwix app + OSMAnd on an old Android phone and auto updating it once a year.

        • clif@lemmy.world
          1·
          2 days ago

          That’s a good question (and good idea) that I hadn’t really thought about past a collection of ZIMs. The one I built advertises it’s own AP SSID that anyone can connect to and then access the ZIMs that are served via kiwix-serve on HTTP/80. That is, I wanted a single, low power, headless device that multiple people could use simultaneously via wifi and browser rather than a personal device.

          I hadn’t really thought about other helpful services past that. I mean, we’ve got a (wee) server so why not use it? I like the idea of OSM and their website is open source but has a lot of dependencies :

          openstreetmap-website is a Ruby on Rails application that uses PostgreSQL as its database, and has a large number of dependencies for installation

          A fully-functional openstreetmap-website installation depends on other services, including map tile servers and geocoding services, that are provided by other software. The default installation uses publicly-available services to help with development and testing.

          I wonder how hard it would be to host everything it needs locally/offline… and what that would do to power consumption : )

          Thanks for the idea - something to look into, for sure.

          • Fmstrat@lemmy.world
            1·
            2 days ago

            I might beat you to it. I’ve got Kiwix running in docker, just did a PR to the kiwix-zim-updater so it can run in Docker on a cron schedule next to the server, and have spun those up with Karakeep (self-hosted web archive I use for bookmarking).

            Right now I’m adding a ZIM list feature to the updater to list available ZIMs by language, and then I’ll move on to OSM.

            • clif@lemmy.world
              1·
              2 days ago

              Yeah, I feel the same in that it’s assuredly doable, but how hard is it?

              If you’re able to dig into and make some progress, please tag me because I’m interested but don’t have much time these days.

    • mistermodal@lemmy.ml
      29·
      3 days ago

      Yeah also if you make a Zim wiki or convert a website into Zim then you can run that stuff too. If you use Emacs it’s easy to convert some pages to wikitext for Zim too

    • Fmstrat@lemmy.world
      2·
      2 days ago

      120GB not including Wikimedia 😉

      Also, I wish they included OSM maps, not just the wiki.

    • Random Dent@lemmy.mlEnglish
      1·
      2 days ago

      You can also offline the whole of Project Gutenberg with Kiwix, it’s about 70GB IIRC.

    • Gigasser@lemmy.world
      1·
      3 days ago

      I wonder if there’s anyways to edit these files afterwards? They tend to be read only, right? I must confess, I don’t have too much experience with this myself.

      • Prathas@lemmy.zip
        4·
        2 days ago

        It’s probably hundreds of thousands of HTML files, no? What is the fear about being able to edit or not?

          • Prathas@lemmy.zip
            1·
            2 days ago

            Okay, I’m unfamiliar with both. Well, I still don’t understand why read-only state matters; are you concerned about tampering?

            • Gigasser@lemmy.world
              1·
              21 hours ago

              Well I think it would be cool to be able to fix/edit any inaccurate articles, or pages that may have been messed with by trolls, or to update with more up to date info.