Charles Elkan, Goldman Sachs’ global leader in machine learning, continues to focus on technology and automation on the first day of VentureBeat’s Transform 2020 digital conference and is one of today’s many leading AI leaders. His fireside chat offered specific instructions on how to use AI in the financial world. Though AI isn’t ready to replace people, Elkan has a unique ability to provide actionable guidelines based on large amounts of data ̵
Given his previous experience as ML Director of Amazon and a professor at UC San Diego, Elkan was very familiar with time series forecasts that have traditionally been based on historical data – if possible for years – to predict future needs. With modern ML, including deep learning neural networks, Elkan says that 52-week forecasts can be made for products that are almost brand new. Natural language processing is used to find similar products by searching catalog descriptions and then examining sales trends for those products to derive how a new version works. Amazon has made its forecasting tools generally available through AWS in the past year and promises up to 50% more accurate forecasts than conventional systems.
Elkan also provided a clear guide to developing machine-based products. He identified the key as a product manager, which serves first as the voice of the user when working with developers and then as the voice of the developer when talking to users of the product. The main task of the product manager is to focus on the user problem that the ML is supposed to solve and to ensure that the overall product delivers this solution appropriately.
On the way to the final product, the manager needs to understand the size of the opportunity of the ML solution, determine what type of output is useful for the company, and obtain high-quality, useful data for training the machine. The manager must not only understand how the solution fits the company’s existing systems, but also quantify latency, processing volume, and system-level requirements, and then help design the user interface, create guard rails, and monitor the system ongoing use.
ML products can fail for a number of reasons, Elkan said, particularly due to volume, input, output, and perception problems. An ML-based solution could be asked to make decisions that either go beyond a machine’s capabilities – such as the wisdom of venture capital investments – or are within scope, but less the prediction accuracy, to be useful. It may also have no access to the required real-time or historical data, or may be subject to unrealistically high standards compared to current alternatives. Referring to a packet sniffer dog as a biological neural network that was trained for a specific task, Elkan demonstrated that those involved only feel comfortable with a basic explanation of how a biological ML model works, but may have a different standard before use Apply machine solutions for expected comfort or understanding.
During a “flash round” of questions from Wing Venture Capital partner Rajeev Chand, Elkan was asked whether ML models can predict stock prices better than traders and analysts – “sometimes yes” – and critically how distortions can be eliminated. We can eliminate and quantify the prejudices we know, Elkan says, but the challenge is to become aware of the prejudices that we may not be thinking about so that we can consciously remove them. ML and AI bias has been simmering for years, but has recently garnered much more attention thanks to some particularly embarrassing examples and late corporate interest after protests against Black Lives Matter.
Elkan’s public questions included how financial data can be collected in an increasingly privacy-driven world – which he believes has been regulated by strict legal guidelines that Goldman carefully follows – and whether corporate fraud is now being used by criminals to AI attacked, what Elkan said is something that is already happening. Elkan was also asked if Goldman customers were actually asking about AI, and he quickly said yes: in addition to external customers asking about AI solutions, he also works with customers within the company who are curious and looking for useful AI.