Signals

Why Everyone Is Underestimating Reasoning Models

November 28, 2024 - LLM Reasoning, LLM Impacts
1 mins

Researchers are sharing encouraging early reports about o1-preview as an aid for tackling complex scientific challenges.

Researchers at the national lab have also been surprised by o1-preview’s ability to recognize when it doesn’t have all the necessary information to answer a question and make reasonable assumptions for variables it might be missing, the person said.

The Lawrence Livermore example is similar to the positive reaction Australian-American mathematician Terence Tao shared after the initial release of o1-preview and o1-mini. Tao used the models to solve math problems and write proofs—something that a typical ChatGPT user probably wouldn’t do.

“It may only take one or two further iterations of improved capability” until such a reasoning model becomes a “competent graduate student…at which point I could see this tool being of significant use in research level tasks,” he said.

This dilemma mirrors one potentially faced by junior lawyers: if AI handles graduate-level research tasks, how will the next generation of researchers develop their skills?

The Information

Related

How AI Could Break the Career Ladder

Generative AI is increasingly able to perform entry-level work in white-collar jobs, which will impact workforces far into the future.

Across the white-collar economy, entry-level jobs are suddenly vulnerable to automation because they involve low-stakes assignments of the sort that generative AI is best at. AI could therefore sever the career ladder of industries like finance and law, forcing many would-be bankers and lawyers to look elsewhere for work.

Having used some of the more recent AI meeting tools as a meeting scribe, I can attest to the fact that they are increasingly able to perform routine tasks with a good degree of accuracy. Compared to the early days when we’d have a good laugh at the software’s attempts to summarise, it’s now a useful tool for taking notes and extracting action items.

Consider the legal field. Law is among the industries most exposed to generative AI’s capabilities because of its orientation toward language. Traditionally, the first few years of a newly accredited lawyer’s career is spent working under the tutelage of more senior lawyers and engaged in routine tasks—missives like “document review,” basic research, drafting client communications, taking notes, and preparing briefs and other legal documents. Advances in AI-powered legal software have the potential to create vast efficiencies in these tasks, enabling their completion in a fraction of the time—and a fraction of the billable hours—that it has historically taken junior lawyers and paralegals to complete them.

If we don’t need to train up junior lawyers, how do we grow the legal workforce? Or do we need to rethink the role of a lawyer?

Bloomberg
LLM Impacts
November 17, 2024