The Fed had cited concerns about weakening global growth and recent stock market volatility.
The FTSE 100 fell 1.34%, Frankfurt's Dax plunged 3.06%, and in Paris the Cac 40 dropped 2.56%.
On Wall Street, stocks were sharply lower, with the Dow Jones down nearly 1% in morning trading.
Brenda Kelly, head analyst at London Capital Group, said: "Markets have taken cues from the US, but uncertainty prevails and choppiness is the only certain result. Deflation is a concern. China is a concern, and oil prices look set to take another leg lower."
Some analysts said the Fed could still raise rates this year.
"The Fed's assessment of the global economic conditions has made investors nervous as uncertainty about the timing of a US rate hike continues. We think that a rate hike could still be announced in December," said Robert Parkes, equity strategist at HSBC.
But Michael Hewson, chief strategist at CMC Markets, said: "The Fed's rather downbeat outlook came as an unwelcome surprise, and it's likely to take a while for investors to figure out whether the Fed is seeing something that the rest of us aren't."