Burglars broke into De Beers and Omega at the Royal Exchange in the City on Saturday night.
City of London police said they did not yet know the value of the stolen jewellery.
A spokesman said: "Just after 10pm last night De Beers and Omega were broken into at the Royal Exchange and a burglary took place.
"Police are now investigating the incident."
The Royal Exchange, built in 1565, is located between Threadneedle Street and Cornhill and boasts more than 30 luxury stores.
A security guard working at the imposing white stone building said staff had been told the robbers entered through the back of the property.
The man, who did not want to be named, said: "We've been told that at about 10pm last night a BMW turned up at the back entrance, four men jumped out and managed to get inside the building somehow.
"At the moment we don't know how much has been taken, but with the price of some of the stuff inside these shops just a few items could easily run up to a very, very large amount.
"It's looking as though whoever has done this has been targeting the shops for a long time."