At which point does AI make mistakes? Uncovering the mechanisms of judgment through games

LANGUAGE ≫ Japanese

MATSUZAKI Kiminori

Specialized field: Game Informatics, Deep Reinforcement Learning, Programming Methdology

For Details

Today, artificial intelligence (AI) outperforms humans not only in the worlds of Go and shogi, but across diverse fields including text generation and image recognition. Yet the black box problem, i.e., the difficulty of explaining why AI reaches its conclusions, remains unsolved. Furthermore, now that AI can arrive at answers more accurately than humans, fundamental questions have emerged: What counts as a correct answer? How should we evaluate performance? Professor Kiminori MATSUZAKI is tackling these difficult problems mathematically through familiar games. Using the once-popular smartphone puzzle game "2048" as his subject, he's working to visualize, in an easy-to-understand way, how AI learns and at which point it goes wrong.

Toward a world of uncertainty that reflects real life

Professor Matsuzaki's research career originally began with parallel programming: coordinating multiple computers to solve problems at high speed. There, the main research targets were theory and optimization. But after joining KUT in 2009, he began exploring new directions.

"Highly abstract parallel computing tends to feel difficult and distant for students. I wondered if there might be a new theme that students could engage with while actually enjoying themselves. That's when I turned my attention to games. I thought that games, with their clear rules and quantifiable results, would be ideal both as an entry point for learning and as a foundation for research," he recalls.

This led him to shift toward game informatics--an approach that treats games not merely as entertainment, but as a research platform for accurately evaluating AI performance and visualizing its thought processes and problems.

When he joined KUT, global research had already made progress with perfect information games like shogi and Go, where all board information is visible. So Professor Matsuzaki turned his attention to the world of imperfect information games: games involving chance, like dice, or hidden information, like not being able to see your opponent's cards in a card game.

"At that time in 2009, shogi research was already quite mature, and Go research had also advanced considerably. That's why I deliberately targeted an underexplored area. How do you make judgments under conditions of incomplete and uncertain information? That's a theme that relates directly to real-world society."

Building a fully-analyzed, ideal experimental proving ground

To handle this aspect of uncertainty more quantitatively, he chose "2048," a globally recognized puzzle game. In this game, players slide tiles up, down, left, or right on a 4x4 board, combining matching numbers to create larger ones. Because the positions and values of new tiles are determined randomly, the game demands both luck and strategy.

Professor Matsuzaki and his team decided to analyze AI decision-making mechanisms in detail using Mini 2048, a version that shrinks the game to a 3×3 board. They first undertook the work of theoretically solving the game itself. They identified every possible game state that could arise, calculated the optimal move for each situation and the expected score from that point on. They determined the true evaluation values for a massive number of states, approximately 41 million in total. This yielded a database that comprehensively catalogued the expected score for each game state.

This complete analysis established an ideal experimental environment where AI judgments could be verified against true correct answers. During the process of reinforcement learning, AI learns optimal behavior by assigning its own evaluation values, making judgements like "this situation is good" or "that situation is bad." By individually comparing these scores against the true values, it became possible to quantitatively grasp when, where, and to what degree the AI was making incorrect judgments.

"Most game AI research uses final win-loss records or average scores as metrics, but I want to see what's happening along the way. I want to be able to explain 'why did it make that judgment?'" he says.

Based on the complete analysis data, they created a "perfect player" that selects the optimal move in every situation. When they analyzed 10,000 plays, they discovered multiple difficult points where survival rates dropped significantly. They also obtained a clear relationship when testplay was done while adding artificial noise to the true value data: the greater the error, the lower the score. This supported the view that the accuracy of the evaluation function--how AI evaluates each game state--is the determining factor in AI performance.

Next, they compared AI players trained with different structures and conditions using the "N-tuple network," which has been widely used for both 2048 and Othello. They confirmed that even with different settings, error tendencies were largely consistent, and errors increased particularly in the endgame. The "endgame wall" that AI struggles with emerged clearly.

This research showed that there is still considerable room for improvement in learning the evaluation function that determines AI performance.

Why does AI become overconfident?

Once the foundation for measuring AI judgment was in place, the next challenge that emerged was the difficulty of balancing exploration and exploitation. The more actively you try unknown moves, the easier it is to make new discoveries, but errors also increase. Conversely, if you only repeat familiar approaches, learning plateaus. How to control this dilemma is an important theme in the world of reinforcement learning.

To address this, state-of-the-art 2048 players have employed "optimistic initialization," which assigns high initial values to unknown game states to encourage exploration. However, when this was verified with Mini 2048, it became clear that while it is effective at promoting early-stage exploration, its effectiveness is limited. The AI was overconfident in its own predictions and kept repeating incorrect judgments.

Professor Matsuzaki and his team therefore shifted to a design that promotes exploration while emphasizing quality and timing. When they carefully introduced methods such as prioritizing moves that had seldom been chosen in the past, learning stabilized and scores improved. They overturned the conventional wisdom that "exploration is unnecessary in 2048 and overconfident AI performs best," presenting a new perspective that adding appropriate exploration can improve AI learning outcomes.

Furthermore, they newly focused on the issue of overestimation through comparison with true evaluation values using Mini 2048. Efficiency techniques that exploit symmetry, such as board rotation and reflection, are employed in developing 2048 players. However, it was discovered that this was producing unexpected side effects. Under symmetry, especially in the early game where score differences are small regardless of which move is chosen, AI tends to readily assume that "all choices are correct." When these conditions overlap, AI becomes overconfident in its own "success experiences," causing overestimation.

As a solution to this problem, Professor Matsuzaki and his team introduced a technique called Double Q-learning, in which two independent networks mutually correct each other's learning to suppress overestimation. The experimental results confirmed that overestimation was indeed suppressed, but this time they observed the opposite tendency: underestimation, where evaluations shifted lower overall. The AI began avoiding risks and making decisions that erred on the side of caution. In other words, they had created an AI that was too cautious. This is an unsuitable property for combining with conventional game tree search.

When overconfidence is corrected, the AI becomes timid instead, and there is great significance in being able to quantitatively visualize this fluctuation on the solid foundation of complete analysis. The biases in judgment latent within AI's learning process are becoming more concretely graspable.

Reexamining the conventional wisdom of AlphaZero, the world's strongest game AI

Having thus unraveled each of AI's thinking processes, Professor Matsuzaki now sets his sights on examining the reasons behind the strength of AlphaZero, the world's strongest game AI developed by Google DeepMind.

AlphaZero is known for achieving strength far surpassing top professionals in shogi, Go, and chess through an innovative method that fuses three technologies: reinforcement learning, neural networks, and Monte Carlo tree search (MCTS). However, the basis for its strength has not yet been clearly explained.

"AlphaZero is extremely good, but there has been almost no research that tackles head-on the question of 'why is it strong?' We want to verify its structures one at a time based on theoretical foundations."

Professor Matsuzaki's series of analyses using Mini 2048 have shown that search does not necessarily lead in the correct direction, and that controlling bias in the evaluation function is the key that determines performance. This is a result that clarifies one aspect of the established theory that has been treated as AlphaZero's design philosophy, i.e., that "the combination of neural networks and MCTS is the strongest."

"AlphaZero's developers have explained that 'neural networks perform nonlinear computations, so combining them with MCTS is optimal.' But no one has actually verified whether that's truly the only optimal solution. I want to clarify that 'why.'"

Alongside this fundamental research, Professor Matsuzaki continues pursuing the challenge of creating the world's strongest 2048 player. The current world record stands at 625,000 points. His laboratory's AI player has already reached 587,000 points, putting them on the verge of breaking the record.

"Research that develops theory is important, but when there's a practical goal of aiming to be number one in the world, student motivation really shoots up," he says with a laugh.

Elucidating AI's theoretical issues and striving to break the world record... By pursuing these two approaches in parallel, this research is advancing in a deeper and more certain direction.

Striving for game AI that benefits society

Professor Matsuzaki's research is a challenge that transcends the framework of games to probe the very foundations of AI intelligence. How can rational decisions be made in situations where the future is uncertain, as in the real world? He continues to pursue the answer to that question within the small world of games.

"In most positions in 2048, choosing any direction won't cause major problems, as long as you don't make an obvious mistake. However, in the endgame when the board fills up with tiles, a single misjudgment immediately leads to "game over." In other words, while the game is usually simple, sometimes there are extremely difficult positions. This structure closely resembles the real world we live in. For example, in autonomous driving, most driving is monotonous operation, but instant judgment is required when a child suddenly darts into the street. In real-world society too, most decisions are simple, but it's the occasional difficult points that matter most."

Choosing the best option from multiple alternatives in medical settings, deciding on actions with limited information during disasters, or instantly assessing risk in financial markets... These "rare but extremely important moments" are where AI's true value is tested.

Professor Matsuzaki currently serves as Center Head at the newly-established Research Center for Evolving Game AI Applications at KUT, where he is envisioning new application domains that connect AI with society.

"After thoroughly investigating the principles of AI through complete analysis, my next aim is 'games that benefit society.' Real-world problems can also be reframed as games once you establish rules and scoring systems."

By collaborating with researchers both inside and outside the university, he will seek to apply the game framework and translate complex phenomena into forms AI can understand, thereby connecting his work to solving social issues. The application areas he has in mind are fields like materials chemistry and neuroscience.

"For example, if, in materials science, you treat physical constraints as game rules and evaluate material properties as scores, the problem of finding optimal materials has the same structure as playing a game for a high score. In other words, you can teach AI to treat finding a light yet strong material as winning a game. Similarly, insights about when AI makes mistakes are valuable in neuroscience research that recreates brain function as mathematical models."

It becomes possible to design AI with more human-like learning processes based on the laws of failure learned from games. The crucial point here is eliminating beforehand the AI pitfalls that have been identified thus far.

"By understanding patterns where AI tends to fail in specific situations and designing around them, we can obtain more reliable results. In other words, removing these risks in advance is essential for social implementation. I believe that AI characteristics derived from familiar games will definitely be helpful for supporting human society in the future."

Within the seemingly simple game of Mini 2048, Professor Matsuzaki explores the essence of AI intelligence and connects those insights to solving social issues. His research not only reexamines why AI is powerful, but also strives to open up possibilities for AI to evolve into something more trusted within human society--something we can make decisions together with.

Date of posting: January 2026 / Date of interview: October 2025