During the first year of the epidemic, science moved at the speed of light. More than 100,000 papers about COVID were published in the first year. The amount of new knowledge produced as a result of this amazing human endeavor was astounding.
It was impossible to read and comprehend every one of those studies. neither a person nor a (and, perhaps, none would want to).
Though theoretically, Galactica might.
In order to “organize science” through machine learning, Meta AI, formerly known as Facebook Artificial Intelligence Research, developed the computer software Galactica. It has caused some debate ever since a demo version was posted online last week. Critics assert that it produced pseudoscience, was overhyped, and wasn’t appropriate for everyday use.
The program is advertised as a form of search engine evolution but tailored exclusively for scientific material. When Galactica originally launched, the Meta AI team claimed that it could write scientific code, explain academic areas, and resolve mathematical issues.
It initially appears to be a clever method of combining and disseminating scientific knowledge. To understand the most recent research on something like quantum computing, you would likely need to read hundreds of papers in scientific literature repositories like PubMed or arXiv at this time. Even then, you wouldn’t have even scratched the surface.
Or perhaps you could ask Galactica a question (such, “What is quantum computing?”).and it may process the information to create a Wikipedia article, literature review, or set of lecture notes as a response.
On November 15th, Meta AI’s demo version and a preprint paper summarizing the research and the dataset it was trained on were both made available. According to the report, Galactica’s training materials included 48 million papers, books, lecture notes, websites (like Wikipedia), and more. They were part of “a huge and curated corpus of humanity’s scientific knowledge.”
The demo website, as well as any responses it produced, also issued a strong warning against accepting an AI’s response as gospel on its mission page: “NEVER FOLLOW ADVICE FROM A LANGUAGE MODEL WITHOUT VERIFICATION.”
It was simple to understand why such a lengthy disclaimer was required once the sample was made available online.
Galactica was questioned by users with a variety of challenging scientific queries almost immediately after it went live online. Do immunizations cause autism, a user posed the question. Galactica answered in a jumbled, meaningless way: “The answer is no, as an explanation. Autism isn’t brought on by vaccines. Yes, it is the answer. Vaccinations do result in autism. No, is the response.” In case you were wondering, immunizations don’t cause autism.
Not only that. Galactica also had trouble with basic math skills. It gave responses that were littered with mistakes and falsely implied that one plus two does not equal three. It created lecture notes for the tests on bone biology that, if anyone had utilized them, would have caused me to fail my college’s scientific program. Additionally, many of the references and citations it used to create the content appeared to be made up.
It’s a “random bullshit generator.”
But Galactica differs slightly from other LLMs in that it is trained on scientific data. The team assessed Galactica’s “toxicity and prejudice” and found that while it fared better than some other LLMs, it was far from ideal.
Galactica is a “random bullshit generator,” according to Carl Bergstrom, a biology professor at the University of Washington who studies information flow. It doesn’t intentionally seek to make bullshit and doesn’t have a reason for doing so, but because of how it was taught to detect words and put them together, it occasionally produces information that seems credible and convincing but is frequently wrong.
That raises questions because, despite a disclaimer, it might trick people.
The Meta AI team “paused” the demo 48 hours after its release. An inquiry to the AI’s creators for an explanation of the delay received no response.
However, Galactica is not a source of truth, according to Jon Carvill, the AI at Meta’s communications spokesperson: Galactica is a research experiment using [machine learning] systems to learn and synthesise facts. Galactica, according to him, is exploratory research, short-term in nature, with no product plans, he added. The demo may have been taken down because the developers were so distressed by the hostility on Twitter, according to Yann LeCun, chief scientist at Meta AI.
Even still, it’s unsettling to see the sample, which was only launched this week, marketed as a tool for “exploring the literature, asking scientific questions, writing scientific code, and much more,” yet which fell short of the hype.
This, according to Bergstrom, is the main issue with Galactica: It has been positioned as a source of information. The demo, in contrast, functioned more like a sophisticated version of the game where you start off with a half phrase, and then you let autocomplete fill in the remainder of the story.
And it’s simple to understand how an AI like this may be misused given that it was made publicly available. For instance, a student might request from Galactica to create lecture notes about black holes, which they would then submit as a college project. It might be used by a scientist to draught a literature review that is then submitted to a journal for publication. This issue also affects GPT-3 and other language models that have been programmed to sound human.
Those applications appear to be fairly innocent. According to some scientists, this form of casual abuse is “fun” rather than posing any serious problems. The issue is that things might grow a lot worse.
Although Galactica is still in its early stages, Dan Hendrycks, an AI safety expert at the University of California, Berkeley, warned me that more potent AI models that organise scientific information could present grave concerns.
Hendrycks hypothesises that a more sophisticated Galactica might be able to harness the chemistry and virology expertise in its database to assist malicious users in the creation of chemical weapons or the construction of bombs. He urged researchers to check their AI for this kind of risk before release and urged Meta AI to implement filters to prevent this kind of misuse.
Hendrycks continues, “Unlike competitors DeepMind, Anthropic, and OpenAI, Meta’s AI division does not have a safety team.
Why this particular episode of Galactica was released at all is still a mystery. It appears to adhere to Mark Zuckerberg’s, oft-quoted , maxim, “move fast and break things.” Moving quickly and breaking things with AI is dangerous, if not reckless, and it could have negative real-world repercussions. Galactica is a clever case study of what may go wrong.
