AIs Evolve – And Not Always for the Better. A recent paper explored the performance of OpenAI’s ChatGPT 3.5 and 4 over multiple release cycles – and found that the models change pretty dramatically in their performance (e.g., GPT-4 performed very well in a test to identify prime numbers in March ’23 but flubbed the test in June). As these models are completely opaque black boxes, the findings highlight the necessity for continuous performance monitoring when using proprietary models.