Business secretary Sajid Javid has acknowledged that the £130 million tax settlement with Google was "not a glorious moment" for the government.
Last week, Chancellor George Osborne labelled the deal as a "major success". But Mr Javid said the settlement has left the country with a "sense of injustice", intensifying claims that huge companies receive preferential treatment.
Speaking on the BBC's Andrew Marr Show, the business secretary said: "I speak with thousands of companies small, medium-sized as well as, of course, large companies and there is a sense of injustice in what they see."
The £130m settlement amounts to a corporation tax rate of just 3% over several years. It was later revealed that Italy secured a bigger tax deal than the UK, despite Google having earned less in Italy.
However, Mr Javid said that the deal could be viewed as a "success" in that it will change the behaviour of other large businesses.
Google told the Andrew Marr Show that it would like to see the tax system updated.
Peter Barron, senior Google communications director, said: "We would like to see more simplicity and more clarity, not least because we would like to be seen to be paying the right amount."