Anthropic, a public benefit company specializing in artificial intelligence (AI), has unveiled Claude 2, marking a significant milestone in the rapidly evolving field of generative AI. By introducing Claude 2, Anthropic demonstrates notable enhancements in various domains, including near-instantaneous query response times and the ability to process inputs as large as 100,000 tokens.
In contrast to traditional machine learning benchmarking approaches, Anthropic took a novel approach to evaluate Claude 2’s capabilities. The researchers conducted head-to-head tests between Claude and its successor, focusing on real-world knowledge, skills, and problem-solving abilities. Claude 2 outperformed its predecessor across the board, excelling in knowledge, coding, and other examinations. Surprisingly, it even achieved commendable results when compared to human averages, specifically scoring above the 90th percentile on GRE reading and writing exams and performing similarly to the median applicant in quantitative reasoning. However, experts caution against direct comparisons between AI and human test takers due to disparities in cognitive reasoning and potential biases within the training datasets.
Alongside the launch of Claude 2, Anthropic introduced a beta version of a web-based interface called “Talk to Claude.” This interface allows users in the U.S. and U.K. to access the chatbot, providing a convenient platform for interaction.
Cointelegraph conducted preliminary testing of Claude 2 and observed immediate improvements. The new model responded promptly to prompts, delivering concise and clear answers. Notably, the prompt limit for Claude 2 is set at 100,000 tokens, equivalent to approximately 75,000 words. While users should be able to upload various document types such as PDF, TXT, and CSV for parsing, this functionality did not function during the limited testing conducted by Cointelegraph.