Ask HN: Why isn't all stock trading done by algorithm?

In the case of the long term, however, trading is done with fundamental indicators. These things can be more or less intangible and have to do with market events, people, and other indicators of company value that are hard to translate into math. Using fundamental indicators for portfolio management is what humans are better at, and these pay off in the long term (see Warren Buffet). These trades are done with skill, which, as I stated earlier, is exponentially more effective at creating gains than breadth.

In short, it takes a huge amount of breath to get the gains required by a relatively small amount of skill. Computers are better by far at breadth, while humans are better (for now) at skill. This, I think, is why humans still trade.

Stock traders trade on rumor, fact and everything in between. They (can) look at who the company is run by, and how much they trust him, and look at the people in upper management. They can look at and understand news.

Part of the difficulty of the stock market is that it isn't a closed system: people make decisions to sell stocks based on the fact that they are poor this month. Until computers can understand and process all the external data, people will need to be in the trading loop.

The big exception to this is the trading the noise: high frequency traders trade against each other, closing the spread in bid/ask prices. However, they are essentially trading against other computers at that timescale, which makes them pretty amenable to algorithms.

For any given trading strategy, a ton of thought, testing, and domain knowledge goes into creating the algorithm. It is not a black box that writes itself.

That said, computers are far more effective at certain tasks, especially latency sensitive simple calculations, just as calculators are far better at doing arithmetic.

Renaissance Technologies founder Jim Simmons is famous for saying they didn't override the algorithm.

In practice most HFT firms do a mix of both. The algos will do the vast bulk of the trading but you have human traders monitoring algos to do clean ups for cases where the algo gets "stuck". What defines "stuck" really depends on the sophistication of the algo and the firm itself.

Some algo's, such as internalizers's for crossing bought flow are so simple that there probably doesn't need to be much over site at all.

Market making is very similar, with the exception of a flash crash where they might pull out, market making algos should just run themselves.

Source: I work in electronic trading and have the past ~7+ years.

There's also a big difference between "trading" and "investing." Trading is what you've described -- buying shares in the morning and hoping they'll go up in the afternoon so you can sell them later. Investing is buying shares of the company to become a part owner and hold them for years or decades, not days.

If you looked at NFLX's chart in 2012, you could "discern" that the share price would continue to hover around that price, maybe go up a little, maybe go down a little. And you could have bought it in September 2012 for $8 a share and sold for a nice $1 profit in October 2012 for $9 a share (split adjusted). But what the chart wouldn't have told you -- and would never have been able to tell you -- is that it would skyrocket in 2013 and up to its current split-adjusted price of $110 a share. The thing is, the chart never would have told you about this. And even a "pure" numerical analysis like could be done by a computer -- P/E ratios, cash flows, etc -- would not have predicted that growth. You could do DCFs all day every day in 2012 and never predict Netflix's rise. There are a lot of things that go into a company's rise that aren't numerical, like the quality of management, market moat, market growth, etc. And, of course, you had to buy it, and hold it for years, in order to see that return.

(In the interests of full disclosure, I should probably note that I'm a bit biased in this. I work for The Motley Fool, which advocates for long-term buy and hold investing, and produce a podcast for growth investors called Rule Breaker Investing.)

The short answer to your question is: because most of the volume traded on exchanges is large blocks of stock being bought and sold by institutional investors, and you need humans to make these deals happen.

Longer explanation as follows:

On the trading floor [0], you have two groups of people: the sales team and the traders.

The sales team gets paid when they make markets, i.e., connect buyers and sellers. Specifically, the financial institution takes a fee that's a very small % of the overall transaction volume, and some of that goes into the sales team's bonus pool. The more stock trades flow through the firm (specifically, their business unit), the more they get paid.

The traders, on the other hand, gets paid to do two things, which are really the same thing: a) not put too much of the firm's capital at risk and b) set the firm up to make money by buying securities low and selling them high. Every trader has a "P&L" (profit & loss) number, which is the total amount of money they've made or lost for the firm since the start of the fiscal year. They get paid a bonus based on this number. They tend to know exactly what their number is at any given time.

So, there is actually a lot of tension on the trading floor between the sales team and the traders, because the sales team wants a lot of volume to go through their business unit, and any given trader wants to maximize her P&L.

Real world example might be: the sales dude gets a call from a hedge fund saying, "we want to sell $100mm of our shares in Alphabet at $720". He then shouts over to the trader (who sits close to him) to tell her about the call and she thinks for a couple seconds [1] and then says, "you need to make a market for 80mm of those shares at that price, I'm only taking 20mm."

In other words, the trader is saying that she'll only tie up $20mm of the firm's capital on this particular trade [2]. The sales person might come back and say, "c'mon, they took that $50mm of Microsoft stock you were trying to get rid of last quarter, we as a firm owe them a favor" to which the trader might respond, "OK, we'll take $30mm tops". So then the sales person will get on the phone and start calling everyone (other mutual / hedge funds, pension funds, etc) who might be interested in buying Alphabet at $720. Maybe the sales person makes it happen; maybe they don't. In any case, they need to figure out whether they can get 70 million dollars worth Alphabet stock pre-sold to other people in the market at $720 before they get back to the hedge fund trying to sell it with a response as to whether they can make the trade.

All of this involves MASSIVE HUMAN FACTORS. I'm sure we will one day be able to train AI to work through various constructs of "we owe them a favor" but right now you still need humans to get big trades like this done. And again, big trades like this constitute the majority of the overall volume in the market. So, that's why trades don't run entirely on algorithms...yet.

[0] I was in banking, not S&T, but have a decent understanding of how this works.

[1] Being able to make decisions of this magnitude in a couple of seconds (and have them be good ones) is one of the two skills you need to have as a trader; the other one is not letting the outcome of the last trade (good or bad) affect your thinking on the next trade.

[2] There is potential for both upside and downside in a decision like this; if the stock appreciates, the firm can profit by selling the stock at a higher price than it paid, but the reverse is also true. This is also an example of why "proprietary trading" is such a blurry line. In order to make markets for big trades, firms usually have to put their own capital at risk, even for a few minutes. At what point are they trading for their own profit vs. temporarily assuming risk in order to broker a deal between two counterparties? Go read Matt Levine's archived columns at Bloomberg if you find stuff like this interesting.

If it includes where and how to place the trades (a smart order router) I would say 99% of trades are managed by an Algo. Also there are the "dumb" VWAP, POV, TWAP algos which represent the bulk of "smart" buyside money as internally most firms use vwap as benchmark.

The bulk of retail orders in the US are on the other side of an algo from citadel, knight or citi. And often buying 100 shares of MSFT at market only really needs a decent SOR to provide BestEx.

Block trading still often gets fed to VWAP algos unless the stock is illiquid.

Finally the most interesting execution algos (implementation shortfall algos) are hard to explain and only statistically outperform.

If you are talking about actual investing - the first question is which asset class in which to park your money. If you can get an algo to do that (like Renaissance) you will be rich.

How does an algorithm interpret e.g. an Apple Keynote? By the time Twitter sentiment analysis (if such thing is really useful) gives results, an human trader already took a position...

Not to mention the various macro variables, like wars, weather, crime, etc... Plus, what time frame should the algorithms trade on? They're very good for predicting the very short term, I haven't seen much evidence that they're good for predicting longer time spans.

While algos eat up arbitrage and electronic brokers replace human ones, humans are still very good at other forms of trading... Not to mention, a large part of market activity isn't even trading - it's long term investing and collecting dividends.

Finally, don't forget that somebody has to design and write the algorithms.

I used to be a value investor around 2000-2008. A value investor would be something like Buffet or Peter Lynch. However I did make a lot money in Sept. 08 because I determined the market was over valued.

What I didn't forsee, was how much the dollars the Federal reserve would print and inflate the economy.

Regardless, after that I built my own algorithm, because I no longer believe in the structure of the market. I would rater trust numbers. Meaning there are to many analyst pumping stocks, federal reserve, insider trading, spoofing trades, ETFs, deratives, and financial warfare it's hard to make a true value investment. Yes, I have read the buffet / Grahm books, but those are over ~60 old.

I think it is Virtu (electronic trading / hedge fund) that hasn't had a day where they lost money since early 2009? I know Goldman and JP Morgan 90% of the time trade every day for a profit. So a lot of the market is already trading electronically. I think zerohedge.com has estimated the 70% of the market trades on electronically and that article was few years ago.

It's funny, because I have devised methods using social media / programming to manipulate the price of stocks. If I can think of ways to do that I'm sure sure Wall St. already is doing it.

Anyways here my algorithm it tracks over 500 stocks: http://www.strategic-options.com/trade/

All the hedge-fund managers that claim to have algorithmic trading have extremely high expense ratios. So its cheaper if I made trades myself.

I mean, its only $7 to execute a trade off of Scottrade or E-Trade. While buying a mutual fund with algorithmic trading will cost you like 50 to 200 basis points per year.

Yeah, its cheaper to trade in the raw or to just buy SPY or Vanguard funds (which are passively invested without algorithms)

So at the extreme (high frequency trading), all trading is indeed done by machines.

If you discovered that the market always goes up on Tuesday and drops on Wednesday that only works until everyone else discovers the same thing and starts selling on Tuesday and buying on Wednesday.

It's simple economics. If they wouldn't be able to make money they probably wouldn't trade.

- Lack of sophistication. "Classically" trained finance people don't know much about computers. I took a finance class at a top business school, and it's nothing compared to Engineering. A bit of time-value-of-money and maybe some option math, but really it doesn't come close to the sophistication of a CS or Engineering course. I went to a meeting last week with a guy who wanted an automated trading system. He hadn't heard of Python. He didn't have any idea how to execute other than on 3rd party programs (which of course use algos, but he was just providing the decisions).

- Lack of scale. There's a lot of family offices who have a few tens to hundreds of millions of dollars. If they wanted an algo trading guy, they'd have to pay him a lot of money, you'd want more than one, and you'd need infrastructure. Plus there's the risk you get all this, hire the guys, and their results are no better than random. A lot of small fortunes like this tend to spend more time in tech-soft areas, like private equity or private debt. The stock trades are an afterthought that they can't spend much resource on.

- Two kinds of decision making: arbitrage and investment. The put it bluntly, arbitrage is easy to mechanize. If some guy quotes some options at the wrong value, it's obvious you want to trade with him. There's looser arbs (things that sort of always come back to normal), but the principle is the same. In some sense, it's not a financial challenge, it's a technological one. For investment (I think XYZ corp will go up), you need to have a sense of what risk you want to take. Utility functions are not easy to put into code. You can try, but you end up with situations where you decide not to have the algo on. There's also the principal-agent problem; most traders are agents, they need to look good to their boss. They need to be able to explain why they are betting on some company. Often, more effort goes into how to justify your trades than what trades to do.

- Things that can't go into a machine: I worked with a guy who used to go meet the CEOs, look them in the eye, and ask them if they'd make money. Now I'm not saying this approach works, but if this is your investment edge, how are you ever going to put that in a machine?

- Insider information: taking this in the loose sense, not the criminal one. If you're highly dependent on understanding some part of the market better than others, you may be better off talking and networking rather than coding. Goldmans are great at this. Every time you meet them, they offer a bit of info in exchange for yours. It lets them see things like the mortgage bubble before it happens, whereas a model would probably have issues due to the small amount of computerized data.