For the reasonings of any men whatsoever, even though they be Catholics, and of high reputation, are not to be treated by us in the same way as the canonical Scriptures are treated. We are at liberty, without doing any violence to the respect which these men deserve, to condemn and reject anything in their writings, if perchance we shall find that they have entertained opinions differing from that which others or we ourselves have, by the divine help, discovered to be the truth. I deal thus with the writings of others, and I wish my intelligent readers to deal thus with mine. (A Letter of Instructions (Commonitorium) to the Holy Brother Fortunatianus, AD 415)
But who can fail to be aware that the sacred canon of Scripture, both of the Old and New Testament, is confined within its own limits, and that it stands so absolutely in a superior position to all later letters of the bishops, that about it we can hold no manner of doubt or disputation whether what is confessedly contained in it is right and true; but that all the letters of bishops which have been written, or are being written, since the closing of the canon, are liable to be refuted if there be anything contained in them which strays from the truth, either by the discourse of some one who happens to be wiser in the matter than themselves, or by the weightier authority and more learned experience of other bishops, by the authority of Councils; and further, that the Councils themselves, which are held in the several districts and provinces, must yield, beyond all possibility of doubt, to the authority of plenary Councils which are formed for the whole Christian world; and that even of the plenary Councils, the earlier are often corrected by those which follow them, when, by some actual experiment, things are brought to light which were before concealed, and that is known which previously lay hid, and this without any whirlwind of sacrilegious pride, without any puffing of the neck through arrogance, without any strife of envious hatred, simply with holy humility, catholic peace, and Christian charity? (On Baptism, Against the Donatists, Book 2, Chapter 3, ca. AD 400)
As regards our writings, which are not a rule of faith or practice, but only a help to edification, we may suppose that they contain some things falling short of the truth in obscure and recondite matters, and that these mistakes may or may not be corrected in subsequent treatises. For we are of those of whom the apostle says: "And if ye be otherwise minded, God shall reveal even this unto you." Such writings are read with the right of judgment, and without any obligation to believe. In order to leave room for such profitable discussions of difficult questions, there is a distinct boundary line separating all productions subsequent to apostolic times from the authoritative canonical books of the Old and New Testaments. The authority of these books has come down to us from the apostles through the successions of bishops and the extension of the Church, and, from a position of lofty supremacy, claims the submission of every faithful and pious mind. If we are perplexed by an apparent contradiction in Scripture, it is not allowable to say, The author of this book is mistaken; but either the manuscript is faulty, or the translation is wrong, or you have not understood. In the innumerable books that have been written latterly we may sometimes find the same truth as in Scripture, but there is not the same authority. Scripture has a sacredness peculiar to itself. In other books the reader may form his own opinion, and perhaps, from not understanding the writer, may differ from him, and may pronounce in favor of what pleases him, or against what he dislikes. In such cases, a man is at liberty to withhold his belief, unless there is some clear demonstration or some canonical authority to show that the doctrine or statement either must or may be true. But in consequence of the distinctive peculiarity of the sacred writings, we are bound to receive as true whatever the canon shows to have been said by even one prophet, or apostle, or evangelist. Otherwise, not a single page will be left for the guidance of human fallibility, if contempt for the wholesome authority of the canonical books either puts an end to that authority altogether, or involves it in hopeless confusion. (Reply to Fautus the Manichaean, AD 400)
However, if you inquire or recall to memory the opinion of our Ambrose, and also of our Cyprian, on the point in question, you will perhaps find that I also have not been without some whose footsteps I follow in that which I have maintained. At the same time, as I have said already, it is to the canonical Scriptures alone that I am bound to yield such implicit subjection as to follow their teaching, without admitting the slightest suspicion that in them any mistake or any statement intended to mislead could find a place. (From Augustine to Jerome, Chapter 3) (A letter from Augustine to Jerome, AD 405)
From these quotations, it would appear that St. Augustine did indeed believe in Sola Scriptura. Scripture is clearly, for him, in a unique position, compared to other writings. Augustine sounds here quite a lot like later Protestant theologians on this subject.
But does he say anything here that Catholics, who certainly do not believe in Sola Scriptura, could not say? Do Catholics believe that Scripture is unique compared to the other writings of men? Yes. Catholics hold that the writings of Cyprian, of Augustine, of Ambrose, and of every other father, are not infallible, but that Scripture is. So it may seem like Augustine is deciding for Protestantism against Catholicism on this point, but if we look more carefully, he is actually only affirming here what both sides would agree on.
If we look further in Augustine, however, we will see that he does indeed decide between the Protestant and the Catholic view on the matter of Sola Scriptura:
"The apostles," indeed, "gave no injunctions on the point;" but the custom, which is opposed to Cyprian, may be supposed to have had its origin in apostolic tradition, just as there are many things which are observed by the whole Church, and therefore are fairly held to have been enjoined by the apostles, which yet are not mentioned in their writings. (On Baptism, Against the Donatists, Book 5, Chapter 23, ca. AD 400)
But the admonition that he [Cyprian] gives us, "that we should go back to the fountain, that is, to apostolic tradition, and thence turn the channel of truth to our times," is most excellent, and should be followed without hesitation. (On Baptism, Against the Donatists, Book 5, Chapter 26, ca. AD 400)
As to those other things which we hold on the authority, not of Scripture, but of tradition, and which are observed throughout the whole world, it may be understood that they are held as approved and instituted either by the apostles themselves, or by plenary Councils, whose authority in the Church is most useful, e.g. the annual commemoration, by special solemnities, of the Lord's passion, resurrection, and ascension, and of the descent of the Holy Spirit from heaven, and whatever else is in like manner observed by the whole Church wherever it has been established. (Letter to Januarius, AD 400)
As to the first man, the father of mankind, it is agreed by almost the entire Church that the Lord loosed him from that prison; a tenet which must be believed to have been accepted not without reason,--from whatever source it was handed down to the Church,--although the authority of the canonical Scriptures cannot be brought forward as speaking expressly in its support, though this seems to be the opinion which is more than any other borne out by these words in the book of Wisdom. (Letter to Evodius, AD 414)
27. On the question of baptism, then, I think that I have argued at sufficient length; and since this is a most manifest schism which is called by the name of the Donatists, it only remains that on the subject of baptism we should believe with pious faith what the universal Church maintains, apart from the sacrilege of schism. And yet, if within the Church different men still held different opinions on the point, without meanwhile violating peace, then till some one clear and simple decree should have been passed by an universal Council, it would have been right for the charity which seeks for unity to throw a veil over the error of human infirmity, as it is written "For charity shall cover the multitude of sins." For, seeing that its absence causes the presence of all other things to be of no avail, we may well suppose that in its presence there is found pardon for the absence of some missing things.
28. There are great proofs of this existing on the part of the blessed martyr Cyprian, in his letters—to come at last to him of whose authority they carnally flatter themselves they are possessed, while by his love they are spiritually overthrown. For at that time, before the consent of the whole Church had declared authoritatively, by the decree of a plenary Council, what practice should be followed in this matter, it seemed to him, in common with about eighty of his fellow bishops of the African churches, that every man who had been baptized outside the communion of the Catholic Church should, on joining the Church, be baptized anew. And I take it, that the reason why the Lord did not reveal the error in this to a man of such eminence, was, that his pious humility and charity in guarding the peace and health of the Church might be made manifest, and might be noticed, so as to serve as an example of healing power, so to speak, not only to Christians of that age, but also to those who should come after. For when a bishop of so important a Church, himself a man of so great merit and virtue, endowed with such excellence of heart and power of eloquence, entertained an opinion about baptism different from that which was to be confirmed by a more diligent searching into the truth; though many of his colleagues held what was not yet made manifest by authority, but was sanctioned by the past custom of the Church, and afterwards embraced by the whole Catholic world; yet under these circumstances he did not sever himself, by refusal of communion, from the others who thought differently, and indeed never ceased to urge on the others that they should "forbear one another in love, endeavoring to keep the unity of the Spirit in the bond of peace." For so, while the framework of the body remained whole, if any infirmity occurred in certain of its members, it might rather regain its health from their general soundness, than be deprived of the chance of any healing care by their death in severance from the body. And if he had severed himself, how many were there to follow! What a name was he likely to make for himself among men! How much more widely would the name of Cyprianist have spread than that of Donatist! But he was not a son of perdition, one of those of whom it is said, "You cast them down while they were elevated;" but he was the son of the peace of the Church, who in the clear illumination of his mind failed to see one thing, only that through him another thing might be more excellently seen. "And yet," says the apostle, "show I unto you a more excellent way: though I speak with the tongues of men and of angels, and have not charity, I have become as sounding brass, or a tinkling cymbal." He had therefore imperfect insight into the hidden mystery of the sacrament. But if he had known the mysteries of all sacraments, without having charity, it would have been nothing. But as he, with imperfect insight into the mystery, was careful to preserve charity with all courage and humility and faith, he deserved to come to the crown of martyrdom; so that, if any cloud had crept over the clearness of his intellect from his infirmity as man, it might be dispelled by the glorious brightness of his blood. For it was not in vain that our Lord Jesus Christ, when He declared Himself to be the vine, and His disciples, as it were, the branches in the vine, gave command that those which bare no fruit should be cut off, and removed from the vine as useless branches. But what is really fruit, save that new offspring, of which He further says, "A new commandment I give unto you, that you love one another?" This is that very charity, without which the rest profits nothing. The apostle also says: "But the fruit of the Spirit is love, joy, peace, long-suffering, gentleness, goodness, faith, meekness, temperance;" which all begin with charity, and with the rest of the combination forms one unity in a kind of wondrous cluster. Nor is it again in vain that our Lord added, "And every branch that bears fruit, my Father purges it, that it may bring forth more fruit," but because those who are strong in the fruit of charity may yet have something which requires purging, which the Husbandman will not leave untended. Whilst then, that holy man entertained on the subject of baptism an opinion at variance with the true view, which was afterwards thoroughly examined and confirmed after most diligent consideration, his error was compensated by his remaining in catholic unity, and by the abundance of his charity; and finally it was cleared away by the pruning-hook of martyrdom. (On Baptism, Against the Donatists, Book 1, Chapter 18, AD 400)
To be sure, although on this matter, we cannot quote a clear example taken from the canonical Scriptures, at any rate, on this question, we are following the true thought of Scriptures when we observe what has appeared good to the universal Church which the authority of these same Scriptures recommends to you; thus, since Holy Scripture cannot be mistaken, anyone fearing to be misled by the obscurity of this question has only to consult on this same subject this very Church which the Holy Scriptures point out without ambiguity. (Against Cresconius, found here, taken from Robert B. Eno, Teaching Authority in the Early Church [Wilmington: Michael Glazier, 1984], p. 134)
Now that all Faustus' calumnies have been refuted, those at least on the subjects here treated of at large and explained fully as the Lord has enabled me, I close with a word of counsel to you who are implicated in those shocking and damnable errors, that, if you acknowledge the supreme authority of Scripture, you should recognise that authority which from the time of Christ Himself, through the ministry of His apostles, and through a regular succession of bishops in the seats of the apostles, has been preserved to our own day throughout the whole world, with a reputation known to all. (Reply to Fautus the Manichaean, AD 400)
And if any one seek for divine authority in this matter, though what is held by the whole Church, and that not as instituted by Councils, but as a matter of invariable custom, is rightly held to have been handed down by apostolical authority, still we can form a true conjecture of the value of the sacrament of baptism in the case of infants, from the parallel of circumcision, which was received by God's earlier people, and before receiving which Abraham was justified, as Cornelius also was enriched with the gift of the Holy Spirit before he was baptized. (On Baptism, Book 4, Chapter 24, AD 400)
2. For if the lineal succession of bishops is to be taken into account, with how much more certainty and benefit to the Church do we reckon back till we reach Peter himself, to whom, as bearing in a figure the whole Church, the Lord said: "Upon this rock will I build my Church, and the gates of hell shall not prevail against it!" The successor of Peter was Linus, and his successors in unbroken continuity were these:--Clement, Anacletus, Evaristus, Alexander, Sixtus, Telesphorus, Iginus, Anicetus, Pius, Soter, Eleutherius, Victor, Zephirinus, Calixtus, Urbanus, Pontianus, Antherus, Fabianus, Cornelius, Lucius, Stephanus, Xystus, Dionysius, Felix, Eutychianus, Gaius, Marcellinus, Marcellus, Eusebius, Miltiades, Sylvester, Marcus, Julius, Liberius, Damasus, and Siricius, whose successor is the present Bishop Anastasius. [This is the succession of the Bishops of Rome down to Augustine's day.] In this order of succession no Donatist bishop is found. But, reversing the natural course of things, the Donatists sent to Rome from Africa an ordained bishop, who, putting himself at the head of a few Africans in the great metropolis, gave some notoriety to the name of "mountain men," or Cutzupits, by which they were known.
3. Now, even although some traditor had in the course of these centuries, through inadvertence, obtained a place in that order of bishops, reaching from Peter himself to Anastasius, who now occupies that see,--this fact would do no harm to the Church and to Christians having no share in the guilt of another; for the Lord, providing against such a case, says, concerning officers in the Church who are wicked: "All whatsoever they bid you observe, that observe and do; but do not ye after their works: for they say, and do not." Thus the stability of the hope of the faithful is secured, inasmuch as being fixed, not in man, but in the Lord, it never can be swept away by the raging of impious schism; whereas they themselves are swept away who read in the Holy Scriptures the names of churches to which the apostles wrote, and in which they have no bishop. For what could more clearly prove their perversity and their folly, than their saying to their clergy, when they read these letters, "Peace be with thee," [1713] at the very time that they are themselves disjoined from the peace of those churches to which the letters were originally written? (Letter to Generosus, AD 400)
So, upon closer and more careful examination, it seems that St. Augustine was no friend of Sola Scriptura after all. He did indeed extoll highly the value and uniqueness of Scripture. He did indeed put it in a higher and more authoritative place than the other writings of men. But--and here is the key--he never put Scripture in opposition to the Church and its Tradition, and he always took Scripture along with the interpretations and the Tradition of the Catholic Church. Compare St. Augustine's attitude with the very different attitude of Martin Luther over a thousand years later:
Unless I am convinced by Scripture and plain reason - I do not accept the authority of the popes and councils, for they have contradicted each other - my conscience is captive to the Word of God. I cannot and I will not recant anything for to go against conscience is neither right nor safe. God help me. Amen.
Luther, like Augustine, extolled the authority and the uniqueness of Scripture. But unlike Augustine, Luther opposed his own interpretation of Scripture to the interpretation and Tradition of the Catholic Church. This huge difference made Luther a Protestant, but Augustine a Catholic.
For more, see here, here, and here.
ADDENDUM 1/27/22: In this link you will find another selection from the writings of St. Augustine which further confirms how his view was definitely not the view of Sola Scriptura.