Jo Johnson said the institutions urgently need to review the "value for money" of some courses after research by Sky News found withdrawal rates of up to two thirds.
:: Sky Data poll: One in three say uni 'not worth it'
Dropout rates per course are not published by universities, but according to data obtained under the Freedom of Information Act:
:: At Wrexham University, a BSc course in Entrepreneurship had a 50% withdrawal rate while 67% dropped out of a degree in Marketing and Consumer Psychology at the same university
:: 67% dropped out a Human Resources Management course at Middlesex University and 28.5% left a BA in Business Management
:: At the University of East London, the dropout rate for a BSc course in Computer Games Development was 58%
:: At Sunderland University, 33% dropped out of a degree in Information Communication Technology
:: At London Metropolitan University, 59% left a BA course in Film, Media and Music
Mr Johnson said: "I urge these universities to take action and look urgently at these courses to ensure that students and the taxpayer are receiving value for money for their investment in higher education.
"Dropout rates are lower now than they were in 2009, but we know there is more work to do."
Requests were made to two dozen universities about how many first-year students withdrew from, rather than failed, their course.
Students who drop out are liable for fees for part of the year, depending on the point they leave. The universities pointed out that some of the highest dropout rates involved very small numbers of students and that there may be a variety of reasons for the decision.
But Lord Adonis, the former Labour education minister and "architect" of the fee rise to £3,000-a-year in 2004, said: "For each individual student who leaves their course early, this is a personal tragedy since they've invested a huge amount of time, emotional energy and ambition into their courses.
"But it's very telling that the universities where individual courses have drop rates of 40, 50, 60% are the same universities are the same universities that have overall drop rates of more than 10 percent. So it's clear that there's something systematically going wrong."
:: Scottish universities under 'financial stress'
Tory MPs are urging action over fees, and the 6.1% interest rates students pay on them, in this autumn's Budget, to try and win back younger voters.
In the aftermath of the election, an insider revealed there had been discussion in Downing Street about abolishing fees altogether.
But the Chancellor, Philip Hammond, is now understood to be standing firm against cutting the interest rates students pay, and to be considering how to pressure universities whose graduates are less likely to get high-skilled jobs.
But ministers are alert to concerns about students' living costs at university, which can contribute to drop-out rates, and the salary threshold at which loans are paid back which has been frozen at £21,000-a-year since 2012.
Universities will soon have to publish data on their applications, drop-out rate and attainment, and prove they are boosting access for disadvantaged students, under new laws aimed at holding institutions to account.
Mr Johnson has already announced that vice-chancellors paid more than the Prime Minister will have to improve their performance.
Tory MP Robert Halfon, former education minister and now chair of Commons Education Commit