Commentary: Combine optimization, machine learning and simulation to move freight

The purpose of this series is to seek evidence to support or refute that premise.

As I stated in the July 7 commentary, Optimal Dynamics is setting out to solve dynamic resource allocation problems, a set of problems that deal with the allocation of scarce resources in an optimal manner over space and time when conditions are uncertain and changing randomly in complex networks.

How did we get here?

Dynamic resource allocation problems are a class of problems that Warren Powell, co-founder of Optimal Dynamics, has studied over the course of his 39-year professorship at Princeton University, where he is a member of the Department of Operations Research and Financial Engineering. As Founder and Manager of Princeton University’s Computational Stochastic Optimization and Learning Labs (CASTLE Labs), Powell has been at the forefront of researching and developing models and algorithms for stochastic optimization with practical applications in transportation and logistics. He will become a professor emeritus at Princeton University effective September 1, 2020.

He co-founded Optimal Dynamics in 2016, with his son Daniel Powell, who is Optimal Dynamics’ CEO.  

If you are a regular reader of FreightWaves, you have encountered discussions of network optimization in supply chain logistics before in this column. For example: Commentary: Toshiba’s simulated bifurcation machines may optimize your supply chain (February 17, 2020); Commentary: Applying machine learning to improve the supply chain (July 30, 2019); Commentary: How can machine learning be applied to improve transportation? (July 23, 2019); and Logistics network optimization – why this time is different (April 23, 2019).

Optimal Dynamics’ platform, CORE.ai, makes the company’s proprietary high-dimensional artificial intelligence, High-Dimensional AI, available for general use through the CORE.ai web portal. It can also be implemented by trucking fleets and by other software vendors that wish to implement it within their products – for example a transportation management system could implement CORE.ai through Optimal Dynamics’ Open API protocols.

Eduardo Silva, Optimal Dynamics’ Vice President of Engineering, says the company’s RESTful API is built on top of a secure, reliable and scalable microservice infrastructure running in the cloud. Customer data is fully encrypted both at rest and in-transit, and Optimal Dynamics has adopted and adheres to best-practice fault-tolerance techniques and uses well-tested tools and strategies to ensure the reliability of the CORE.ai platform while maintaining the highest level of performance as scale increases.

CORE.ai’s High-Dimensional AI uses approximate dynamic programming, a version of reinforcement learning adapted for high-dimensional problems in operations research, based on the insights gained over the decades of research conducted at CASTLE Labs.

Reinforcement learning is a form of machine learning in which the software system learns to accomplish a defined goal by trial-and-error, within a changing environment. Algorithms accomplish this through repetitive feedback loops based on reiterative improvements to a set of available actions. In approximate dynamic programming these available actions are encoded in mathematical functions which are known as policies. In this context, a policy tells the computer model how to act optimally under uncertainty.

Early forms of reinforcement learning, and dynamic programming, were first developed in the 1950s.

Warren Powell explains the difference between reinforcement learning and approximate dynamic programming this way, “In the 1990s and early 2000s, approximate dynamic programming and reinforcement learning were like British English and American English – two flavors of the same algorithmic strategy. Then, as people discovered that this entire algorithm strategy (whether it is ADP or RL) did not solve all problems, people started branching out.”

Don’t worry if this is all starting to sound confusing. He says “These buzz-phrases are so confusing, especially when even the research community is unable to define the terms. Argh!”

What matters is that some of these algorithmic strategies are ready for prime time. As Optimal Dynamics indicates, some of these algorithmic strategies are ready to solve important problems in big, global, legacy industries that are fundamental to our way of life.

The academic research from which CORE.ai is a descendent has been applied in R&D collaborations between CASTLE Labs and large industrial and corporate partners representing every major supply chain logistics subsector.

For example, in Schneider National Uses Data To Survive A Bumpy Economy, which appeared in the September 12, 2011 issue of Forbes, the author describes how a prior version of the technology from CASTLE Labs was applied to create a  “fleet-wide ‘tactical planning simulator’ that would use software algorithms to mimic the decision-making of human dispatchers on an inhumanly large scale.”

Daniel Powell told me that Schneider National credits the technology developed in collaboration with CASTLE Labs with helping it realize $39 million in annual savings at the time.

The Forbes article also describes how other models that were being developed in parallel at CASTLE Labs were implemented by other logistics companies going as far back as the 1980s, and how that transformed the industry even then. For example an interactive freight optimization product called SuperSPIN was used by every major national and regional less-than-truckload (LTL) carrier.

According to CASTLE Lab’s website, “SuperSPIN was a model that arrived during a period of tremendous change in the LTL trucking industry. SuperSPIN allowed companies to understand the trade-offs between the number of end of lines and the value of network density. It also played a role in determining which carriers survived, and was used in the planning of some of the largest LTL carriers that survived the shakeout.”

Manhattan Associates, the publicly traded software company, continues to support SuperSPIN.

Combing through the literature on CASTLE Labs’ website one finds mention of collaborations with, and research funding from other companies like YRC, Ryder Truck Lines, Roadway Package System (now part of FedEx), Embraer, UPS, Netjets, The Air Mobility Command, Air Products and Chemicals, Burlington Motor Carriers, Triple Crown Services, Sea-Land (now part of Maersk), North American Van Lines, The Burlington Northern Santa Fe Railroad (now BNSF) and Norfolk Southern. With Norfolk Southern, CASTLE Labs used approximate dynamic programming to optimize locomotives.

Warren Powell’s Ph.D. dissertation was on bulk service queues for LTL trucking. It was only after he started a new project as a faculty member, with a carrier (Ryder Truck Line) that he learned about the load planning problem, which is an optimization problem.

His Ph.D dissertation was funded by IU International – a diversified services company with interests in trucking, distribution, environmental services, food services and agribusiness, which was acquired in 1988 through a hostile takeover.

Many years ago, a Fortune 500 third-party logistics company built a proprietary network optimization system on predecessors to CORE.ai.

Reflecting on his work in freight transportation, Warren Powell says “My work was roughly split between less-than-truckload – which used one modeling technology, and truckload, rail, Embraer and other operational applications – which used other modeling technologies. They all focused on operational models that required making decisions now that approximated the impact of these decisions on an uncertain future.”

I asked Warren Powell why over the decades during which he has been studying dynamic resource allocation problems, the time is now ripe for Optimal Dynamics to take the work that has been done at CASTLE Labs, to bring it into the real world and to apply it to an entire industry like trucking rather than to discrete, one-off problems within discrete one-off companies.

He said, “The trucking industry has been trying to develop advanced analytics since Schneider National initiated the effort in the late 1970s, but there was always something in the way – lack of data (where is my driver?), poor computing facilities, and a basic lack of the types of analytics required to handle problems in the trucking industry.”

He added, “30 years of research has developed the analytics we need to allow computers to solve these complex problems. This required combining the power of deterministic optimization tools (which emerged in the 1980s and 1990s), with machine learning and stochastic simulation, all at the same time.”

Powell continued, “We can now run these powerful algorithms on the cloud, which offers virtually unlimited computing power. Finally, smartphones and the internet allow us to be in direct touch with drivers, avoiding the need for clumsy telephone calls (1980s) or even the use of expensive satellite systems.”

Today, the path to market for startups like Optimal Dynamics has been somewhat smoothed by the broad awareness among business executives that the technology landscape has changed dramatically.

In the 1980s, Warren Powell’s work was often called the “bleeding edge.” Now, everyone understands the vast power of computers and the cloud, as well as the widespread adoption of smartphones that provide pervasive connectivity and facilitate direct communication with drivers. 

In the past few years, people have also started to realize that computers can be smart through “AI,” although there remains tremendous confusion about what this really means, since AI is actually an entire family of algorithmic technologies. 

According to Warren Powell, the breakthroughs that enable computers to solve chess and Chinese Go simply are not robust enough to optimize a trucking company because of the number of variables that a trucking company must account for, and the uncertainty one must contend with in the real world.

He commented, “It took me a lifetime to realize how to combine the power of optimization [to solve high-dimensional decision problems, but without uncertainty], with machine learning and simulation to crack the high-dimensional problems that arise in freight transportation.”

I have personally been witness to how Warren lights up when he is thinking about how his work applies to problems in logistics and transportation. It happened when I first met him in 2016.

It happened again when I introduced him to executives at the freight forwarding unit of a large European container shipping company in March 2018.

Warren and I met with them at their headquarters, and wound up spending more than four hours talking about freight forwarding and how the various techniques developed at CASTLE Labs could be applied to solving some of the problems they wanted to solve in order to improve their operations. I left Warren with them after hours of conversation (I had a long drive home and wanted to beat traffic). To my amusement, they were so engrossed in conversation that they barely acknowledged that I was leaving.

I came away convinced that a lack of sufficient data would not be as big of an issue as I had previously assumed, and also that the problems the executives described could definitely be solved.

A bit of background

On November 23, 2016 I published Industry Study: Freight Trucking (#Startups). That blog post includes Optimal Dynamics in a very early and rudimentary market map of startups building software for the trucking industry. I came to know Optimal Dynamics and the people behind it after spending a day at CASTLE Labs in August 2016.

Daniel Powell has presented demos of early versions of CORE.ai at The New York Supply Chain Meetup in March 2018: Artificial Intelligence & Supply Chains, and again during The Worldwide Supply Chain Federation’s inaugural global summit, #SCIT19, in June 2019 (Video).

Juliana Nascimento, Optimal Dynamics’ Head of Optimization and Artificial Intelligence, was a panelist at #SCIT19 on the topic of innovation in land-based supply chain logistics (Video). Among other things, Juliana ran Operational Planning & Foreign Trade, and before that Production Planning & Control, and Strategic Planning for eight years at Kimberly-Clark in Brazil, after she earned her Ph.D under the supervision of Warren Powell at CASTLE Labs.

Conclusion

As far as supply chain logistics is concerned, a platform like CORE.ai can be applied in rail, drayage, container shipping, air freight, and warehousing and distribution. Predecessors to CORE.ai have been applied in long-distance and middle-distance trucking, rail and air, real-time dispatching, routing and scheduling, and spare parts management, among others.

Estimates of the market for artificial intelligence in supply chain logistics applications peg the size of the global market at about $6.5 billion by 2023, with a compound annual growth rate of about 43%, according to Infoholic Research. Or, $10 billion by 2025 with a compound growth rate of about 46% according to BizWit Research & Consulting LLP.

As I stated in my July 7 commentary, the goals of this series are: 

• First, to uncover the situations in which artificial intelligence is producing results. 
• Second, to learn why problems in legacy industries pose such a challenge for artificial intelligence systems. 
• And, third, to learn what is required to fulfill AI’s potential in legacy industries.

In the next article in this series we will talk about high-dimensional decision problems, such as the problems encountered in freight logistics, and why they pose such a challenge for AI systems like IBM Watson and Google Deepmind’s AlphaGo.If you are a team working on innovations that you believe have the potential to significantly refashion global supply chains we’d love to tell your story in FreightWaves. I am easy to reach on LinkedIn and Twitter. Alternatively, you can reach out to any member of the editorial team at FreightWaves at media@freightwaves.com.