Little bit of everything!

Avid Swiftie (come join us at !taylorswift@poptalk.scrubbles.tech )

Gaming (Mass Effect, Witcher, and too much Satisfactory)

Sci-fi

I live for 90s TV sitcoms

  • 53 Posts
  • 2.87K Comments
Joined 3 years ago
cake
Cake day: June 2nd, 2023

help-circle
  • This is the only sane answer here, and it makes sense because of the sentiment on Lemmy.

    There is one constant rule about software engineering. You must be adaptable. The career is ever changing, you need to be okay with that. I think a lot of people right now are finding out that if they dig in their heels they think they’re making a point, but the company doesn’t care, there’s the door. AI is just another change in the career. Adapt, or be left behind.

    The job isn’t the same as it was 5 years ago, which also was different than it was 10 years before that, and then 10 years before that. I’ll say this is a large change, but that’s the job.

    I think the biggest thing is there’s no room for “I’m a react engineer” anymore. Everyone needs to be everything, and it means learn as much as you can as fast as you can. You must be a “T-shaped” engineer. Wide breadth, with specific deep knowledge that makes you stand out. You can be an expert at react, but should also know how to code in the backend, and how to deploy, how to work with APIs, some basic cloud architecture. If you’re not learning, you’re falling behind.


  • Car, Power, Electricity, they’re all falling behind. US is doing so many stupid laws right now because it’s the last place that the lobbies are desperately holding onto. Rest of the world has already been moving away, and the companies know it. Now the shareholders are starting to know it. You literally can’t beat free electricity from the sun. They could only prolong it so long, and you can tell they’re desperate with forcing the US to act against it’s best interest.







  • That all makes sense to me, and lines up with what I’ve been reading too. I saw the model download and I was like “guhhhh” to it because I was also excited to try it on my 3090. I’ll be waiting for the quants.

    Yeah I like the end there too, that OpenAI / Anthropic have been desperately trying to figure out how to do this, and a few guys with limited hardware did it. When you have unlimited resources, you end up needing unlimited resources. When you only have 300 GPUs, you make it work. It’s why tech is littered with people starting in garages, they found a way to make it work.




  • Sure, but that’s not my point. Social pressures aside, there is no way for instance a to control what instance (or server, or data collector) B does with it. Unless you audit every server federated, there is no way to know if anyone is doing anything with your data.

    3 letter agencies can and may even already have servers that look like ordinary fediverse servers already just happily listening and storing everything. No amount of social pressure here is going to stop that. So all users need to understand this about the fediverse, that anyone can do whatever they want with your data whenever. For privacy, go to Matrix.


  • I think we’re seeing a lot of optimization right now. The most exciting one I’ve seen is TurboQuant. Short version, every message you send to a model has context, the entire conversation you’ve had, instructions, skills, everything. That takes up an exponential amount of ram, and this is what is causing the VRAM/RAM shortage. TurboQuant (and other copycats now) claims that it can reduce that VRAM usage of the context by 20x. That’s absolutely huge, that’s 1M context models running on consumer hardware potential huge.

    Deepseek v4 also boasts some large claims, saying they have a model that does better than Anthropic’s or OpenAI, while being 1/10th the size. That also is a huge reduction in compute and VRAM, but I’ll be looking for the proof.

    We’ve seen other items too, with upgrades in running models, how quickly results are streamed, to me TurboQuant is the most exciting.

    I think it’s good that they’re finally looking at optimization. Yes, their cost has been power and compute. NVidia is more than happy to keep things inefficient because they sell GPUs that way. Software companies are doing the opposite now, reducing the compute overhead to start saving them money, which they desperately need to do if this is going to continue. New technology has always been horribly inefficient, it’s only once more people see it does it start to get optimized.

    I think this is what is going to be required to finally push past the horribleness of AI companies, and they need to do this quickly.







  • GOOD. NO ONE should be trusted here! I’m just some guy who decided to spin up a server, there should be zero trust! THIS IS MY POINT.

    Don’t you think everyone deserves the information they need to choose which instances they want to interact with, according to whatever criteria is important to them? Even if your criteria are different?

    This depends on the trustworthiness of the admin themselves, and even then every admin is just some person who decided to spin up a server, just like me. Trust is built and earned, it shouldn’t be implicit. The option you have is to defederate, or leave and join another server.

    I’m really not trying to be an asshole here, but your post is what caused me to do this. This is not a unique post, this is a fundamental core principal of the fediverse that every user must understand. That by being here, it is not a private secure place, you are quite literally blasting every comment, post, and upvote, to whoever wants to listen. Literally everyone. Any semblance of privacy is purely a UI trait. Rules/guidance is purely 100% based on what each server owner chooses.


  • and it will continue to be. Again, you need to understand this. There are no rules, guidelines or anything that an instance owner needs to follow beyond whatever legal requirements they have in their specific jurisdiction.

    So, I guess in your pervalence, you are correct, you do not have that freedom. Even I, as an instance owner, do not have that freedom, because everything I’m typing here is being sent out to as many servers are listening too. By being completely open so that anyone can spin up a server and listen for activity, it literally means that we are open and any server can listen for activity.

    Anyone can spin up a server, create some LLM bot, and start replying to anyone they want. That instance can be defederated of course, but that is the only tool. This is what you signed up for, this is the open and free internet. We do not have any walls here.