*2.1.3 Reinforcement learning*

Reinforcement learning is a model that learns to maximize future value instead of giving the correct answer. In other words, it learns to act in a way that maximizes value through trial and error. The problem setting is similar to that of

**Figure 1.** *AI and machine learning relationship diagram.*

supervised learning, but it is not enough to learn the output of the given correct answer as it is, but it is necessary to learn the behavior that maximizes value in a broader sense.

Tetris game is a good example to understand the scheme of reinforcement learning. When playing a game of Tetris, the problem of getting the highest possible score can be considered in the framework of reinforcement learning. The best way to score at that point is to play in such a way that even a single row can be eliminated immediately, but in the longer term, the score will be higher if you accumulate as much as possible and then eliminate many rows at once.

AlphaGo, which defeated a human player, also incorporates reinforcement learning in some parts of its games. As in the case of Go, it can learn even when humans do not necessarily know the correct answer, so it is expected to acquire the ability to surpass humans.

## *2.1.4 Artificial neural network*

A neural network is a mathematical model inspired by the function of nerve cells (neurons) and their connections, or neural networks in the human brain, called artificial neurons. When the neural network model is properly constructed according to the problem to be solved, it can make a variety of decisions (i.e., outputs), such as the following:

First, image recognition and binary classification. For example, it answers questions such as: Is the object in front of me a ship or a train?

Second, natural language processing, multi-level classification. For example: which is "Emma Maersk" among various images of vessels online?

#### *2.1.5 Deep learning*

Deep Learning is a method of machine learning in which neural networks are combined in multiple layers to enhance their representation and learning capabilities. Currently, it is the most commonly used algorithm for AI.

Deep learning, on the other hand, is often used when complex unstructured data is available and is applied in fields such as speech recognition, image recognition, and natural language processing.

There are many cases where conventional machine-learning methods do not work well for classification and regression without complex function approximation, and deep-learning methods are increasingly being used for such problems. In some cases, deep-learning methods have dramatically improved recognition accuracy compared with conventional methods, and deep learning is currently attracting a great deal of attention in the world. Recently, it has been used in a wide range of fields such as recommendation and automated driving.

#### **2.2 Blockchain**

BC is an open distributed ledger technology (DLT) based on a peer-to-peer (P2P) approach that allows transactions to be recorded on thousands of servers simultaneously. On the efficient, verifiable, and immutable BC platform, anyone can see the transactions of others in near real time, making it difficult for one user to manipulate the records and control the network [2]. Applying these features, BC facilitates the digitization of traditional economic, legal, and political systems.


#### **Table 1.**

*Typical consensus algorithms.*

In BC, cryptography is used to store records (hash values10) of transactions that occur in the network in blocks of records called blocks. In each block, it contains three values, the first is the hash value of the previously generated block, the second is the record of the transaction in the current block, and the third is a new hash value generated by a disposable random value called a nonce. The three values are passed to the next block and the accumulated blocks form a chain of blocks in time series. The name BC comes from this data structure.

BC can be divided into public BC, which allows anonymous participation, and permitted BC, which requires permission to participate. Public BC is mainly applied to cryptocurrencies. Since the permitted type of BC is faster than the public type in handling transactions, it has been applied to various business fields such as supply chain and intellectual property management [3].

There have been five main types of algorithms for consensus building in BC (**Table 1**). In public BC, PoW and PoS are mainstream; in the PoW consensus, rewards are evaluated by the amount of work done. In PoW consensus, the reward is evaluated by the amount of work done; in other words, the network participant who performs the appropriate computation the fastest receives the reward. In the PoS consensus, rewards are based on both the amount of work and the amount of cryptocurrencies held. In PoS consensus, rewards are based on both the amount of work done and the amount of cryptocurrencies held, easing the fierce competition in PoWs and saving electricity consumption, it suffers the problem that the rich get richer. On the other hand, in permitted BC, PBFT is the mainstream method. PBFT is faster than other consensus methods but has the disadvantage that blocks will not be created if two-thirds or more of the consensus is not obtained.

<sup>10</sup> A hash value is a fixed length of data created using a hash function. It is unidirectional and is difficult if not impossible to restore the original data from a hash value.

*Shipping Digitalization and Automation for the Smart Port DOI: http://dx.doi.org/10.5772/intechopen.102015*

In the maritime industry, many players from different industries, usually in several countries, are involved in currently operating blockchain platforms. This complexity leads to a lack of transparency in the entire supply chain. In addition, the industry has the disadvantage of high transaction costs for information exchange, the possibility of fraud and theft, and vulnerability to the risk of cyber-attacks. BC offers the possibility to solve these problems [3]. To maximize the capacity and productivity of the digital information space, traditional authentication methods and data structures need to be reformed and modern technologies such as BC need to be actively applied.

#### **2.3 Physical internet (PI)**

PI was initially proposed by Montreuil in 2010 [4]. He defined PI as an open, global logistics network that efficiently and sustainably interconnects all elements of the logistics process. The PI includes the complete supply chain including storage, movement, supply, and delivery of goods, and the PI network is composed of various logistic providers. The goal of PI is to create a global logistics system based on the interconnection of existing logistic networks. To achieve this, a standardized set of protocols, modular containers, and smart interfaces are combined modular containers, called PI containers (**Figure 2**), which come in various sizes and can be combined and loaded to reduce waste. In addition to modular containers, PI-stores, PI-movers, PI-conveyors, and PI-gateways have also been proposed.

The characteristics of PI include interoperability, modularity, and standard interfaces and protocols. In order to take advantage of these features, technologies to share data while protecting confidentiality, such as BC technology, are essential.

Although it is not difficult to understand the usefulness of PI and BC, specific application measures have not been fully studied. In this chapter, we make a proposal for planning measures to build a BC network in PI, and discuss issues and measures for practical application. Specifically, the next sections also aim to clarify the following two points. First, to clarify the scope of application of BC in PI. Second, we propose a framework for implementing PI and BC technologies.

#### **Figure 2.** *An example of PI container combination. Source: Montreuil et al., 2010 [4].*
