Disclaimer: I am not informed about the development of AI, LLMs, etc. or what we were doing with them pre-covid.
I remember them exploding when everyone was doing at home/ hybrid classes, and I wonder how much of an effect on things that really had.
Yes but I’m not sure LLMs would be the focus. Back in 2018-2019 we were already in an AI craze. Investors were throwing billions at it (compared to trillions now).
However LLMs were seen as little more than interesting toys by researchers. Conversational models were secondary compared to image recognition and generation which was considered way more cool than text.
Without spending a lot of time digging into it, I don’t think that the pandemic was a major enabling factor.
Like, work on hardware-driven larger neural nets and generation of images had been going on pre-pandemic. I remember articles about Google researchers working on them quite some time back.
searches
https://en.wikipedia.org/wiki/DeepDream
DeepDream is a computer vision program created by Google engineer Alexander Mordvintsev that uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating a dream-like appearance reminiscent of a psychedelic experience in the deliberately overprocessed images.[1][2][3]
The DeepDream software, originated in a deep convolutional network codenamed “Inception” after the film of the same name,[1][2][3] was developed for the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) in 2014[3] and released in July 2015.
The dreaming idea and name became popular on the internet in 2015 thanks to Google’s DeepDream program. The idea dates from early in the history of neural networks,[4] and similar methods have been used to synthesize visual textures.[5] Related visualization ideas were developed (prior to Google’s work) by several research groups.[6][7]
https://research.google/blog/inceptionism-going-deeper-into-neural-networks/
Like, if you’ve used a diffusion model like Stable Diffusion and run low-iteration stuff in Automatic1111 or ComfyUI or something, that looks pretty familiar.
And that was being released in 2015, half a decade before the pandemic started, and was based on pre-existing work.
I think that the “why now” question mostly just has to do with hardware reaching the point where you can do some significantly-more-interesting things with neural nets, coupled with some mostly-iterative software improvements.
I don’t think that it was a “people are staying inside due to the pandemic and that drove a lot of change” in the sort of sense that, say, there was an impact on video game sales, something that I was talking about on here recently.
EDIT: If you want another data point, I was writing neural-net-based image enhancement software in the early 2000s. Far smaller neural nets than what people are using today for generative AI stuff, but…shrugs
And while I was writing code for CPUs, I distinctly remember a buddy of mine at the time who focused on parallel scientific computing — though he wasn’t working on neural nets — talking about using GPUs to do scientific computation back then, so I know off-the-cuff that there were definitely people banging on that in that timeframe. That’s over twenty years back, and those two things are the fundamental elements behind genAI image stuff.
I used to search out deep dream content for my backgrounds. At the time, I was interested in it because of the psychedelic effects it could create.
I think your timing is wrong. AI exploded after the staying at home was over.
ChatGPT was released on Nov 30, 2022, and that is the point when AI came into mainstream (even if gpt 3 or midjourney had been available already). By that point, stay at home restrictions had been gone by about a year, if not a bit more. For reference, the fifa world cup, with full attendance, started 10 days before
I’ve heard the theory before that the collapse of cryptocurrencies was a larger factor, since you suddenly had lots of cheap GPUs on the market. If anything, I would guess that the pandemic slowed that down, because relatively many people got into gaming and we did have the chip shortage from factories being locked down…
Most of the major developments that lead to the public stuff happened between 2017-2021. Transformers was the big one that made scaling a thing. Altman pushed in a stupid direction that caused a lot of the nonsense, like turning the name “Open AI” into an oxymoron.
There are some aspects of alignment that point at political corruption and planning with nefarious intent that fits in with the present political bullshit too, but that is very complicated to explain in any depth. If you were to search the token vocabulary, you will find dubious elements are present in compound multi word tokens that disproportionately represent a single political camp, likewise with religious media, and science denialism. Much of that stuff dates from 2019 or before.
Contrary to fact hypothesizing.
Pretty much the exact point of stagnation as we currently are.
Where do you think we would be in slavery if the U.S. would have never happened.
Time diversion maybe, but humanities terribleness isn’t dictated by nation nor time period sat out
