The US military was close to an operation against a Chinese ship in the Middle East after an AI-assisted intelligence report contained a false claim about its cargo, CNN reported.
According to the media outlet, it was initially believed that the vessel was carrying components for Iran's nuclear program. However, upon re-examination, it was found that this information was generated using a chatbot and is not true. CNN points out that the report was distributed to US military structures in the spring, when the war with Iran was still underway.
How the mistake was made
According to CNN, an analyst used artificial intelligence to search for cargo information in a ship's manifest. The information received was then summarized again with the help of AI and formatted in a form resembling a standard intelligence document. This is how the false statement acquired the appearance of official analytical material.
According to the sources cited by CNN, the problem was discovered before action was taken against the ship. The media writes that at the time of the re-inspection, US military aircraft were already in the air, and military personnel were also prepared on board for a possible boarding of the vessel.
Questions about AI in the military sphere
The case once again draws attention to the risk of so-called hallucinations in language models, in which the system presents false information as reliable fact. In an intelligence environment, this can lead to erroneous conclusions if there is no human verification of the data.
CNN notes that the US military already uses various artificial intelligence systems to analyze large data sets and help identify targets. That is why the case is seen as a warning about the need for additional control in sensitive operations, where a mistake can have diplomatic and military consequences.
Sources: CNN