Under the Digital Markets Act (DMA), giants such as Google and Apple will be forced to open up their services and platforms to other businesses.
Major technology firms have long faced criticism that they use their market dominance to squeeze out competition.
"What we want is simple: fair markets...in digital," said EU antitrust chief Margrethe Vestager.
"Large gatekeeper platforms have prevented businesses and consumers from the benefit of competitive digital markets," she said.
The announcement is the biggest regulatory move yet from the EU to act against what it defines as "anti-trust" or anti-competitive behaviour from mainly US technology businesses.
"The agreement ushers in a new era of tech regulation worldwide," said German MEP Andreas Schwab, who led the negotiations for the European Parliament.
Under the proposed Digital Markets Act, Apple would be forced to open up its App Store to third-party payment options instead of users being forced to use Apple's own payment system.
It is something Apple has been fighting in the US during a high-profile court battle with Epic Games, the maker of Fortnite.
Google will be asked to offer people who use smartphones which run on the company's Android operating system alternatives to its search engine, the Google Maps app or its Chrome browser.
Apple would also be forced to loosen its grip on the iPhone, with users allowed to uninstall its Safari web browser and other company-imposed apps that users cannot currently delete.
The targets of the law include WhatsApp, Facebook Messenger, iMessage, the App Store, Google Play and many other services belonging to large tech firms.
The EU wants to give users more choice over how people send messages. The new rules would require that technology make their messaging services interoperable with smaller competitors.
However, Apple said it was "concerned that some provisions of the DMA will create unnecessary privacy and security vulnerabilities for our users".
Meanwhile, Google said: "While we support many of the DMA's ambitions around consumer choice and interoperability, we're worried that some of these rules could reduce innovation and the choice available to Europeans."
The law will only affect companies with a value of more than €75bn (£63bn), annual sales of €7.5bn and at least 45 million monthly users.
Legislation was originally proposed by Ms Vestager just over a year ago in reaction to what she felt was monopolistic behaviour from Big Tech. She was known to be frustrated by how mainly large, US tech companies had managed to delay and even thwart EU attempts to fine them.
"The gatekeepers - they now have to take responsibility," Ms Vestager said on Thursday.
Once implemented, the law will give Brussels unprecedented authority in regulating major tech companies.
Many major US tech companies have huge lobbying operations in Washington, and have been emphasising that such laws punish successful American companies.
However many US politicians are also keen to clip the wings of Big Tech, with bills currently going through Congress that would also rein in their power.
With the deal reached by negotiators, the DMA now faces final votes in the European Parliament as well as by ministers from the EU's 27 member states.