David Kupelian, who is not an orthodox Trinitarian Christian, doesn't think much of the book; but didn't say he thought it should be removed. Rather, contra Mark Driscoll's claim, it's one of the least important as opposed to most important books in the canon.
The Protestant-Enlightenment preacher Jonathan Mayhew was accused of a number of things. He wasn't "orthodox" enough for the forces of religious correctness, so he was labeled a "deist" by them. The Song of Solomon features in one of Mayhew's battles with the orthodox. They accused him of wanting to axe it from the canon (and thus demonstrating disrespect for the canon).
I'll quote him below. But if I understand him right, he's say the book "Wisdom" has as much right to belong in the canon as Song of Solomon. And it's not that Song should be out, but rather perhaps Wisdom should be in (both together).
But he goes still further; intimating his suspicions that I am a deist, p. 79.—" The Dr.'s reflection upon the Song of Solomon is sufficient to show how easy it is for him to discard the sacred canon of scripture itself: Or perhaps," &c. But he dared not to cite that refleclion, as he calls it. The most that can be fairly and logically inferred from it, is, that I supposed there was near as much reason for admitting the Wisdom as the Song of Solomon into the canon ;—a very harmless supposition, even tho' it should be a mistake; and which does not imply the latter to be admitted without reason.—Roman Catholics (and the Eastern Orthodox) of course, hold "Wisdom" to be in their canon. They call them deuterocanonical. Protestants call them Apocrypha.