Keeping pace with a rapidly evolving industry like AI is a daunting challenge. Until an AI can do it for you, here’s a handy roundup of recent stories in the world of machine learning, as well as notable research and experiments that we haven’t covered on our own.
This week, Meta released the latest in its Llama series of generative AI models: Llama 3 8B and Llama 3 70B. Capable of parsing and writing text, the models are “open source,” Meta said — intended to be a “fundamental part” of the systems that developers design with their unique goals in mind.
“We believe these are best-in-class open source models, period,” Meta wrote in a blog post. “We embrace the open source philosophy of releasing early and releasing often.”
There is only one problem: the Llama 3 models are not Really “open source”, at least not in the strictest definition.
Open source means that developers can use the models however they want, without hindrance. But in the case of Llama 3 – as with Llama 2 – Meta has imposed certain licensing restrictions. For example, Llama models cannot be used to train other models. And app developers with more than 700 million monthly users must request a special license from Meta.
Debates over the definition of open source are not new. But as AI companies play with the term, it injects fuel into long-standing philosophical arguments.
Last August, a study co-authored by researchers at Carnegie Mellon, the AI Now Institute and the Signal Foundation found that many AI models labeled “open source” have big pitfalls, not just Llama . The data needed to train the models is kept secret. The computing power required to run them is beyond the reach of many developers. And the labor required to fine-tune them is prohibitively expensive.
So if these models aren’t truly open source, what exactly are they? It’s a good question; Defining open source versus AI is no easy task.
A relevant unresolved question is whether copyright, the fundamental intellectual property mechanism on which the open source license is based, can be applied to the different components and elements of an AI project, in particular to the internal scaffolding of a model (e.g., embeddings). Then there’s the disconnect between the perception of open source and how AI actually works: open source was designed in part to ensure that developers could study and modify code without restrictions. With AI, however, the ingredients you need to study and modify are open to interpretation.
Navigating Uncertainty, the Carnegie Mellon Study do highlight the harm inherent in tech giants like Meta co-opting the phrase “open source.”
Often, “open source” AI projects like Llama end up launching news cycles (free marketing) and providing technical and strategic benefits to project managers. The open source community rarely sees these same benefits, and when they do, they are marginal compared to those of maintainers.
Instead of democratizing AI, “open source” AI projects – especially those from big tech companies – tend to consolidate and expand centralized power, the study co-authors say. It’s good to keep this in mind the next time a major release of an “open source” model appears.
Here are some other interesting AI stories from recent days:
- Meta updates its chatbot: Coinciding with the debut of Llama 3, Meta has upgraded its AI chatbot across Facebook, Messenger, Instagram and WhatsApp – Meta AI – with a backend powered by Llama 3. It has also launched new features, including image generation faster and access to web search results.
- AI-generated porn: Ivan writes about how the Oversight Board, Meta’s semi-independent policy council, is turning its attention to how the company’s social platforms handle explicit AI-generated images.
- Instant Watermarks: Social media service Snap plans to add watermarks to AI-generated images on its platform. A translucent version of the Snap logo with a glitter emoji, the new watermark will be added to any AI-generated image exported from the app or saved to the Camera Roll.
- The new Atlas: Hyundai-owned robotics company Boston Dynamics has unveiled its next-generation Atlas humanoid robot, which, unlike its hydraulically powered predecessor, is fully electric and much friendlier in appearance.
- Humanoids on humanoids: Not to be outdone by Boston Dynamics, Mobileye founder Amnon Shashua has launched a new startup, Menteebot, focused on building two-bed robotic systems. A demo video shows the Menteebot prototype walking towards a table and picking up fruit.
- Reddit, translated: In an interview with Amanda, Pali Bhat, CPO of Reddit, revealed that an AI-powered language translation feature to bring the social network to a more global audience is in the works, as well as an online moderation tool. support trained on past decisions and actions of Reddit moderators.
- AI-generated LinkedIn content: LinkedIn has quietly begun testing a new way to increase revenue: a LinkedIn Premium Company Page subscription, which, for a fee that appears to be as high as $99/month, includes AI to write content and a suite of tools to increase the number of followers.
- An indicator : X, the moonshot factory of Google parent Alphabet, this week unveiled Project Bellwether, its latest attempt to apply technology to some of the world’s biggest problems. Here, that means using AI tools to identify natural disasters like wildfires and floods as quickly as possible.
- Protecting children with AI: Ofcom, the regulator responsible for enforcing UK online safety law, plans to launch a study into how AI and other automated tools can be used to proactively detect and remove illegal content online , in particular to protect children from harmful content.
- OpenAI arrives in Japan: OpenAI is expanding into Japan, opening a new office in Tokyo and planning a GPT-4 model optimized specifically for the Japanese language.
More machine learning
Image credits: DrAfter123/Getty Images
Can a chatbot change your mind? Swiss researchers have found that not only can they, but if they are pre-armed with certain personal information about you, they can actually be more more convincing in a debate than a human with the same information.
“It’s Cambridge Analytica on steroids,” said Robert West, project leader at EPFL. The researchers suspect that the model – GPT-4 in this case – drew on its vast stores of online arguments and facts to present a more convincing and confident case. But the result speaks for itself. Don’t underestimate the power of LLMs to persuade, West warned: “In the context of the upcoming US election, people are worried because that’s where this type of technology is still being tested for the first time.” One thing we know for sure is that people will use the power of big language patterns to try to swing the election.
Why are these models so good at language, anyway? This is an area in which there is a long history of research, dating back to ELIZA. If you’re curious about one of the people who was there for a lot of it (and did a lot of it himself), check out this profile on Stanford’s Christopher Manning. He has just received the John von Neuman medal; well done!
In a provocatively titled interview, another longtime AI researcher (who has also graced the TechCrunch stage), Stuart Russell, and postdoc Michael Cohen speculate on “How to stop AI from killing us all.” This is probably a good thing to find out as soon as possible! This isn’t a superficial discussion, though: these are smart people talking about how we can actually understand the motivations (if that’s the right word) of AI models and how Regulations should be built around them.
The interview is actually about a paper in Science published earlier this month, in which they suggest that advanced AIs capable of acting strategically to achieve their goals, which they call “long-term planning agents,” could be impossible to test. Essentially, if a model learns to “understand” the tests it must pass to succeed, it may well learn creative ways to override or circumvent those tests. We’ve seen it on a small scale, why not on a large scale?
Russell is proposing to restrict the materials needed to make such agents… but sure enough, Los Alamos and Sandia National Labs have just received their shipments. LANL has just celebrated the inauguration ceremony of Venado, a new supercomputer for AI research, made up of 2,560 Grace Hopper Nvidia chips.
Researchers are looking into the new neuromorphic computer.
And Sandia just received “an extraordinary brain-based computing system called Hala Point,” with 1.15 billion artificial neurons, built by Intel and considered the largest system of its kind in the world. Neuromorphic computing, as it is called, is not intended to replace systems like Venado, but to seek new methods of computing that are more brain-like than the rather statistics-driven approach we see in modern models.
“With this system of a billion neurons, we will have the opportunity to innovate on a large scale both new AI algorithms that could be more efficient and intelligent than existing algorithms, and new approaches such as brain for existing computer algorithms such as optimization and modeling,” said Sandia researcher Brad Aimone. This looks awesome…just awesome!
techcrunch
