Mimicking the brain: Deep learning meets vector-symbolic AI

Symbolic AI is still relevant and beneficial for environments with explicit rules and for tasks that require human-like reasoning, such as planning, natural language processing, and knowledge representation. It is also being explored in combination with other AI techniques to address more challenging reasoning tasks and to create more sophisticated AI systems. We believe that our results are the first step to direct learning representations in the neural networks towards symbol-like entities that can be manipulated by high-dimensional computing. Such an approach facilitates fast and lifelong learning and paves the way for high-level reasoning and manipulation of objects. Their Sum-Product Probabilistic Language (SPPL) is a probabilistic programming system.

Because machine learning algorithms can be retrained on new data, and will revise their parameters based on that new data, they are better at encoding tentative knowledge that can be retracted later if necessary. Symbolic AI, a branch of artificial intelligence, excels at handling complex problems that are challenging for conventional AI methods. It operates by manipulating symbols to derive solutions, which can be more sophisticated and interpretable.

Exact symbolic artificial intelligence for faster, better assessment of AI fairness

It’s taking baby steps toward reasoning like humans and might one day take the wheel in self-driving cars. They also assume complete world knowledge and do not perform as well on initial experiments testing learning and reasoning. Similar to the problems in handling dynamic domains, common-sense reasoning is also difficult to capture in formal reasoning. Examples of common-sense reasoning include implicit reasoning about how people think or general knowledge of day-to-day events, objects, and living creatures.

This interpretability is particularly advantageous for tasks requiring human-like reasoning, such as planning and decision-making, where understanding the AI’s thought process is crucial. The ultimate goal, though, is to create intelligent machines able to solve a wide range of problems by reusing knowledge and being able to generalize in predictable and systematic ways. Such machine intelligence would be far superior to the current machine learning algorithms, typically aimed at specific narrow domains.

The second AI summer: knowledge is power, 1978–1987

The logic clauses that describe programs are directly interpreted to run the programs specified. No explicit series of actions is required, as is the case with imperative programming languages. But symbolic AI starts to break when you must deal with the messiness of the world.

Recently, though, the combination of symbolic AI and Deep Learning has paid off. Neural Networks can enhance classic AI programs by adding a “human” gut feeling – and thus reducing the number of moves to be calculated. Using this combined technology, AlphaGo was able to win a game as complex as Go against a human being. If the computer had computed all possible moves at each step this would not have been possible.

Learn more about:

As a result, LNNs are capable of greater understandability, tolerance to incomplete knowledge, and full logical expressivity. Figure 1 illustrates the difference between typical neurons and logical neurons. Each approach—symbolic, connectionist, and behavior-based—has advantages, but has been criticized by the other approaches. Symbolic AI has been criticized as disembodied, liable to the qualification problem, and poor in handling the perceptual problems where deep learning excels. In turn, connectionist AI has been criticized as poorly suited for deliberative step-by-step problem solving, incorporating knowledge, and handling planning.

Symbolic artificial intelligence, also known as Good, Old-Fashioned AI (GOFAI), was the dominant paradigm in the AI community from the post-War era until the late 1980s. Deep learning has its discontents, and many of them look to other branches of AI when they hope for the future. As ‘common sense’ AI matures, it will be possible to use it for better customer support, business intelligence, medical informatics, advanced discovery, and much more. McCarthy’s approach to fix the frame problem was circumscription, a kind of non-monotonic logic where deductions could be made from actions that need only specify what would change while not having to explicitly specify everything that would not change.

We show that the resulting system – though just a prototype – learns effectively, and, by acquiring a set of symbolic rules that are easily comprehensible to humans, dramatically outperforms a conventional, fully neural DRL system on a stochastic variant of the game. By integrating neural networks and symbolic reasoning, neuro-symbolic AI can handle perceptual tasks such as image recognition and natural language processing and perform logical inference, theorem proving, and planning based on a structured knowledge base. This integration enables the creation of AI systems that can provide human-understandable explanations for their predictions and decisions, making them more trustworthy and transparent. The Symbolic AI paradigm led to seminal ideas in search, symbolic programming languages, agents, multi-agent systems, the semantic web, and the strengths and limitations of formal knowledge and reasoning systems. Symbolic AI, a branch of artificial intelligence, focuses on the manipulation of symbols to emulate human-like reasoning for tasks such as planning, natural language processing, and knowledge representation. Unlike other AI methods, symbolic AI excels in understanding and manipulating symbols, which is essential for tasks that require complex reasoning.

Henry Kautz,[18] Francesca Rossi,[80] and Bart Selman[81] have also argued for a synthesis. Their arguments are based on a need to address the two kinds of thinking discussed in Daniel Kahneman’s book, Thinking, Fast and Slow. Kahneman describes human thinking as having two components, System 1 and System 2. System 1 is the kind used for pattern recognition while System 2 is far better suited for planning, deduction, and deliberative thinking.

• “Everywhere we try mixing some of these ideas together, we find that we can create hybrids that are … more than the sum of their parts,” says computational neuroscientist David Cox, IBM’s head of the MIT-IBM Watson AI Lab in Cambridge, Massachusetts.
• By integrating neural networks and symbolic reasoning, neuro-symbolic AI can handle perceptual tasks such as image recognition and natural language processing and perform logical inference, theorem proving, and planning based on a structured knowledge base.
• In the CLEVR challenge, artificial intelligences were faced with a world containing geometric objects of various sizes, shapes, colors and materials.

Symbolic AI plays the crucial role of interpreting the rules governing this data and making a reasoned determination of its accuracy. Ultimately this will allow organizations to apply multiple forms of AI to solve virtually any and all situations it faces in the digital realm – essentially using one AI to overcome the deficiencies of another. The tremendous success of deep learning systems is forcing researchers to examine the theoretical principles that underlie how deep nets learn. Researchers are uncovering the connections between deep nets and principles in physics and mathematics. In the CLEVR challenge, artificial intelligences were faced with a world containing geometric objects of various sizes, shapes, colors and materials. The AIs were then given English-language questions (examples shown) about the objects in their world.

Building machines that better understand human goals

Once trained, the deep nets far outperform the purely symbolic AI at generating questions. A second flaw in symbolic reasoning is that the computer itself doesn’t know what the symbols mean; i.e. they are not necessarily linked to any other representations of the world in a non-symbolic way. Again, this stands in contrast to neural nets, which can link symbols to vectorized representations of the data, which are in turn just translations of raw sensory data. So the main challenge, when we think about GOFAI and neural nets, is how to ground symbols, or relate them to other forms of meaning that would allow computers to map the changing raw sensations of the world to symbols and then reason about them. Question-answering is the first major use case for the LNN technology we’ve developed.

“If the agent doesn’t need to encounter a bunch of bad states, then it needs less data,” says Fulton. While the project still isn’t ready for use outside the lab, Cox envisions a future in which cars with neurosymbolic AI could learn out in the real world, with the symbolic component acting as a bulwark against bad driving. It is one form of assumption, and a strong one, while deep neural architectures contain other assumptions, usually about how they should learn, rather than what conclusion they should reach. The ideal, obviously, is to choose assumptions that allow a system to learn flexibly and produce accurate decisions about their inputs. We hope this work also inspires a next generation of thinking and capabilities in AI.

Neurosymbolic AI is also demonstrating the ability to ask questions, an important aspect of human learning. Crucially, these hybrids need far less training data then standard deep nets and use logic that’s easier to understand, making it possible for humans to track how the AI makes its decisions. New deep learning approaches based on Transformer models have symbolic ai examples now eclipsed these earlier symbolic AI approaches and attained state-of-the-art performance in natural language processing. However, Transformer models are opaque and do not yet produce human-interpretable semantic representations for sentences and documents. Instead, they produce task-specific vectors where the meaning of the vector components is opaque.

Binary classification is a type of supervised learning algorithm in machine learning that categorizes new observations into one of two classes. It’s a fundamental task in machine learning where the goal is to predict which of two possible classes an instance of data belongs to. The output of binary classification is a binary outcome, where the result can either be positive or negative, often represented as 1 or 0, true or false, yes or no, etc. Symbolic AI, a subfield of AI focused on symbol manipulation, has its limitations. Its primary challenge is handling complex real-world scenarios due to the finite number of symbols and their interrelations it can process.

We experimentally show on CIFAR-10 that it can perform flexible visual processing, rivaling the performance of ConvNet, but without using any convolution. Furthermore, it can generalize to novel rotations of images that it was not trained for. In conclusion, neuro-symbolic AI is a promising field that aims to integrate the strengths of both neural networks and symbolic reasoning to form a hybrid architecture capable of performing a wider range of tasks than either component alone. With its combination of deep learning and logical inference, neuro-symbolic AI has the potential to revolutionize the way we interact with and understand AI systems. Symbolic AI, a branch of artificial intelligence, specializes in symbol manipulation to perform tasks such as natural language processing (NLP), knowledge representation, and planning. These algorithms enable machines to parse and understand human language, manage complex data in knowledge bases, and devise strategies to achieve specific goals.

The DSN model provides a simple, universal yet powerful structure, similar to DNN, to represent any knowledge of the world, which is transparent to humans. The conjecture behind the DSN model is that any type of real world objects sharing enough common features are mapped into human brains as a symbol. Those symbols are connected by links, representing the composition, correlation, causality, or other relationships between them, forming a deep, hierarchical symbolic network structure. Powered by such a structure, the DSN model is expected to learn like humans, because of its unique characteristics.

It does this especially in situations where the problem can be formulated by searching all (or most) possible solutions. However, hybrid approaches are increasingly merging symbolic AI and Deep Learning. The goal is balancing the weaknesses and problems of the one with the benefits of the other – be it the aforementioned “gut feeling” or the enormous computing power required.