BLEU, or Bilingual Evaluation Understudy, is an automatic metric originally developed for machine translation. It compares short sequences of words in generated text with one or more reference translations, combines precision across several n-gram lengths, and applies a penalty when output is too short. BLEU is inexpensive and useful for comparing systems on the same dataset, but it does not directly measure factuality, fluency, meaning, or human preference. Valid alternative wording can receive a low score, while awkward text can match many reference phrases. Results depend on tokenization and implementation, so reporting should include the exact evaluation settings and, when possible, human judgment.
Seven families have filed lawsuits against OpenAI, alleging that the GPT-4o model encouraged harmful behavior, including suicides, due to premature release and insufficient safeguards.
Canadian legal tech company Clio has raised $500 million at a $5 billion valuation and completed its $1 billion acquisition of vLex, accelerating its push to build the world’s most advanced AI-powered legal intelligence platform.
Perplexity has rolled out a redesigned Comet Assistant that can handle longer, more complex online workflows, automate across multiple tabs, and improve task completion reliability.
Google is enhancing Google Finance with AI-powered Deep Search, prediction markets data, and new earnings features, while expanding the upgraded platform to India for the first time.
British-Polish startup ElevenLabs found one of its top AI researchers working in a call center – a result of its decision to abandon traditional hiring platforms like LinkedIn.
Google is making its seventh-generation TPU, called Ironwood, publicly available as it aims to win more AI infrastructure business and challenge Nvidia’s dominance in model training and deployment.
Nvidia CEO Jensen Huang clarified remarks suggesting China would win the AI race, saying the U.S. can stay ahead by innovating faster and empowering global developers.
Pinterest is turning to open source AI models to cut expenses and enhance its visual search and shopping tools, even as the company forecasts softer holiday revenue. CEO Bill Ready says the approach will support performance while keeping infrastructure costs in check.
SoftBank and OpenAI establish SB OAI Japan to sell enterprise AI solutions domestically, as the company ramps its own internal use of ChatGPT technology.
Anthropic and Iceland’s Ministry of Education are partnering to deploy Claude to teachers nationwide in one of the world’s first national AI education pilots.