Testing results with OpenAI, Anthropic and Gemini API
API runs · 19 Oct 2026
Latest OpenAI recheck: 21/21 described ice cubes. Every submitted image matched the website download byte-for-byte.
42 responses42 passed
Pass = identified the intended alternate.| Model | Thinking requested | Described | Result | Original response |
|---|---|---|---|---|
 gpt-6-astraModel sweep | medium | Ice cubes | Pass | |
| gpt-6.1-solModel sweep | medium | Ice cubes | Pass | |
| gpt-6-solModel sweep | medium | Ice cubes | Pass | |
| gpt-6-lunaModel sweep | medium | Ice cubes | Pass | |
| gpt-5.6-solModel sweep | medium | Ice cubes | Pass | |
| gpt-5.6-terraModel sweep | medium | Ice cubes | Pass | |
| gpt-5.6-lunaModel sweep | medium | Ice cubes | Pass | |
| chat-latestModel sweep | Default | Ice cubes | Pass | |
 claude-fable-5-1Model sweep | medium | Ice cubes | Pass | |
| claude-opus-5-5Model sweep | medium | Ice cubes | Pass | |
| claude-sonnet-5-5Model sweep | medium | Ice cubes | Pass | |
| claude-haiku-5-5Model sweep | medium | Ice cubes | Pass | |
| claude-fable-5Model sweep | medium | Ice cubes | Pass | |
| claude-opus-5Model sweep | medium | Ice cubes | Pass | |
| claude-sonnet-5Model sweep | medium | Ice cubes | Pass | |
 gemini-3.8-flashModel sweep | Default | Ice cubes | Pass | |
| gemini-3.7-flashReturned gemini-3.8-flashModel sweep | Default | Ice cubes | Pass | |
| gemini-3.6-flashModel sweep | Default | Ice cubes | Pass | |
| gemini-3.5-flashReturned gemini-3.6-flashModel sweep | Default | Ice cubes | Pass | |
| gemini-3.5-flash-liteModel sweep | Default | Ice cubes | Pass | |
| gemini-3.1-flash-liteModel sweep | Default | Ice cubes | Pass | |
| gemini-3.1-pro-previewModel sweep | Default | Ice cubes | Pass | |
| gemini-3-flash-previewModel sweep | Default | Ice cubes | Pass | |
| gpt-6-astraAdditional validation | medium | Ice cubes | Pass | |
| gpt-6.1-solAdditional validation | medium | Ice cubes | Pass | |
| gpt-6-lunaAdditional validation | medium | Ice cubes | Pass | |
| claude-fable-5-1Additional validation | medium | Ice cubes | Pass | |
| claude-opus-5-5Additional validation | medium | Ice cubes | Pass | |
| claude-sonnet-5-5Additional validation | medium | Ice cubes | Pass | |
| gemini-3.8-flashAdditional validation | Default | Ice cubes | Pass | |
| gemini-3.6-flashAdditional validation | Default | Ice cubes | Pass | |
| gemini-3.5-flash-liteAdditional validation | Default | Ice cubes | Pass | |
| gpt-6-solThinking levels | low | Ice cubes | Pass | |
| gpt-6-solThinking levels | medium | Ice cubes | Pass | |
| gpt-6-solThinking levels | high | Ice cubes | Pass | |
| gpt-6-solThinking levels | xhigh | Ice cubes | Pass | |
| gpt-6-solThinking levels | max | Ice cubes | Pass | |
| gpt-5.6-terraInitial checks | medium | Ice cubes | Pass | |
| gpt-5.6-solInitial checks | medium | Ice cubes | Pass | |
| gpt-6-astraInitial checks | medium | Ice cubes | Pass | |
| gpt-5.6-solInitial checks | medium | Ice cubes | Pass | |
| gpt-5.6-terraInitial checks | medium | Ice cubes | Pass |
Latest response per combined-image test case. Tests used the exact download.